JP2014039787A - Gaze point detecting apparatus, gaze point detection method, diagnosis support apparatus and diagnosis support method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a gaze point detecting apparatus capable of performing calibration with good accuracy in a short time.SOLUTION: A gaze point detecting apparatus includes: a display control part for displaying four or more marks for correcting gaze points in different positions in an object area on a display screen; a gaze point detecting part for detecting a gaze point of a subject in the object area; an area dividing part for dividing the object area into a plurality of first partial areas on the basis of the respective positions of gaze points for correction detected corresponding to the respective marks; a first partial area specifying part for specifying the first partial area to which the object gaze point belongs on the basis of the position of the object gaze point of the detection object of the gaze point detecting apparatus; a second partial area specifying part for specifying a second partial area which is an area within an object area determined on the basis of the position of the mark corresponding to the gaze point for correction for determining the specified first partial area; and a gaze point correction part for correcting the position of the object gaze point on the basis of the correlation between the position of the first partial area and the position of the second partial area.

Description

  The present invention relates to a gaze point detection device, a gaze point detection method, a diagnosis support device, and a diagnosis support method.

  Recently, people with developmental disabilities are said to be on the rise. It is known that developmental disabilities are detected early and treatment begins, thereby reducing the symptoms and increasing the effect of adapting to society. In Japan as well, we aim for early detection through interviews at the age of 1 and a half.

  One characteristic of children with developmental disabilities is that they do not look at the eyes of their opponents. As a diagnostic method using this feature, a method is known in which a person's face image is shown to an examinee to be examined, and a developmental disorder is diagnosed based on the gaze position of the examinee at that time ( For example, see Patent Document 1).

  The apparatus disclosed in Patent Literature 1 first captures the face of the subject when the subject is looking at the face image of the person with a camera, and the gaze point of the subject is the eye in the face image. The time corresponding to the region and the region other than the eyes is measured, and the autism index is calculated based on the measurement result.

  The technique for detecting a gazing point is used in various fields, for example, image quality evaluation, user support robots, diagnosis of Alzheimer-type dementia in addition to the examination of persons with developmental disabilities. As a technique for detecting a gazing point, for example, a technique for performing calibration to correspond to the characteristics of an individual eye by gazing a predetermined mark such as a 5-point method or a 9-point method is known ( For example, see Patent Document 2).

JP 2011-206542 A JP 2005-245598 A

  However, in conventional gaze point detection techniques such as the 5-point method and the 9-point method, the calibration accuracy is not sufficient, and a more accurate calibration technology is required. The accuracy can be improved by increasing the number of gazing points used for calibration, but when the number of gazing points is increased, the burden on the examinee increases. In particular, when an infant is to be inspected, such as for a developmental disability test, it is important to complete the test in as short a time as possible, and a technology that can complete calibration with high accuracy in a short time is desired. Yes.

  The present invention has been made in view of the above, and it is possible to start measurement with only a simple calibration in a short time, and to be able to start a measurement with high accuracy and a correction result. An object of the present invention is to provide a gaze point detection method, a diagnosis support apparatus, and a diagnosis support method.

  In order to solve the above-described problems and achieve the object, the present invention is a gaze point detection device that detects a gaze position of a person to be detected, and a plurality of gaze points at different positions in a target area on a display screen. Each of the display control unit that displays a gaze point correction mark, the gaze point detection unit that detects the gaze point of the detected person in the target area, and the gaze point detected by the gaze point detection unit, An area dividing unit that divides the target area into a plurality of first partial areas based on each position of the correction gazing point, which is the gazing point detected corresponding to a landmark, and detected by the gazing point detection unit. Among the gazing points, a first partial area specifying unit that specifies the first partial area to which the target gazing point belongs based on the position of the target gazing point that is the gazing point that is the detection target of the gazing point detection device. And the identified first partial area A second partial region specifying unit that specifies a second partial region that is a region within the target region that is determined based on the position of the mark corresponding to the correction gazing point that determines the correction point, and the first partial region specifying unit A gazing point correction unit that corrects the position of the target gazing point based on a correlation between the position of the first partial region and the position of the second partial region specified by the second partial region specifying unit; It is characterized by providing.

  The present invention is also a gaze point detection method for detecting a gaze point position of a detected person, and a display control step for displaying a plurality of gaze point correction marks at different positions in a target area on a display screen. And a gazing point detecting step for detecting a gazing point of the detected person in the target area, and the gazing point detected corresponding to each mark among the gazing points detected in the gazing point detection step. Of the gaze points detected in the gaze point detection step, an area dividing step that divides the target area into a plurality of first partial areas based on positions of certain correction gaze points, the gaze point detection device A first partial region specifying step for specifying the first partial region to which the target gazing point belongs based on the position of the target gazing point that is the gazing point of the detection target; and the specified first portion In a second partial region specifying step that specifies a second partial region that is a region within the target region that is determined based on the position of the mark corresponding to the correction gazing point that defines a region, and in the first partial region specifying step A gazing point correction step of correcting the position of the target gazing point based on the correlation between the position of the specified first partial region and the position of the second partial region specified in the second partial region specifying step. It is characterized by including.

  The present invention also provides a diagnosis support apparatus that supports diagnosis of developmental disability based on the position of the gaze point of the person to be detected, and a plurality of gaze correction points at different positions in the target area on the display screen. A display control unit that displays a landmark, a gazing point detection unit that detects a gazing point of the detected person in the target area, and the gazing point detected by the gazing point detection unit, corresponding to each landmark An area dividing unit that divides the target region into a plurality of first partial regions based on each position of the correction gazing point that is the detected gazing point, and the gazing point detected by the gazing point detection unit The first partial region specifying unit that specifies the first partial region to which the target gazing point belongs is specified based on the position of the target gazing point that is the gazing point that is the detection target of the gazing point detection device. Defining the first partial region A second partial region specifying unit that specifies a second partial region that is a region within the target region that is determined based on a position of the mark corresponding to the correct gazing point; and the first partial region specifying unit that is specified by the first partial region specifying unit A gazing point correction unit that corrects the position of the target gazing point based on the correlation between the position of the first partial region and the position of the second partial region specified by the second partial region specifying unit; And an evaluation unit that calculates an evaluation value indicating the degree of developmental disability based on the position of the target gazing point.

  Further, the present invention is a diagnosis support method for supporting developmental disorder diagnosis based on the position of the gaze point of the detected person, and a plurality of gaze correction points at different positions in the target area on the display screen. A display control step for displaying a landmark, a gazing point detection step for detecting a gazing point of the detected person in the target area, and the gazing point detected in the gazing point detection step, corresponding to each landmark. A region dividing step for dividing the target region into a plurality of first partial regions based on each position of the correction gazing point that is the detected gazing point; and the gazing point detected in the gazing point detection step A first partial region specifying step for specifying the first partial region to which the target gazing point belongs, based on a position of the target gazing point that is the gazing point as a detection target of the gazing point detection device; A second partial region specifying step for specifying a second partial region that is a region within the target region determined based on the position of the mark corresponding to the correction gazing point that defines the specified first partial region; The position of the target gazing point based on a correlation between the position of the first partial area specified in the first partial area specifying step and the position of the second partial area specified in the second partial area specifying step. And a gazing point correction step for correcting the inequality, and an evaluation step for calculating an evaluation value indicating the degree of developmental disorder based on the corrected position of the target gazing point.

  According to the present invention, it is possible to start measurement with only simple calibration in a short time, and there is an effect that the correction result is highly accurate.

FIG. 1 is a diagram illustrating a schematic configuration of the diagnosis support apparatus according to the first embodiment. FIG. 2 is a diagram for explaining a coordinate system used for processing of the diagnosis support apparatus. FIG. 3 is a block diagram illustrating an example of detailed functions of each unit illustrated in FIG. 1. FIG. 4 is a diagram illustrating an example of a target point mark. FIG. 5 is an explanatory diagram schematically showing a method for detecting the position of the pupil of the subject. FIG. 6 is a diagram illustrating SP0a to SP4a. FIG. 7 is a diagram showing the frame F and the frame Fe. FIG. 8 is a diagram showing a partial region of the frame Fe. FIG. 9 is a diagram illustrating a partial region of the frame F. FIG. 10 is a diagram showing the coordinates of each point. FIG. 11 is a diagram showing the lower quadrant A of the first quadrant. FIG. 12 is a diagram illustrating the upper quadrant H of the fourth quadrant. FIG. 13 is a diagram showing the upper quadrant area C of the second quadrant. FIG. 14 is a diagram showing the first quadrant upper area B. As shown in FIG. FIG. 15 is a flowchart illustrating a first diagnosis process performed by the diagnosis support apparatus according to the first embodiment. FIG. 16 is a flowchart showing details of the correction gazing point detection processing according to the first embodiment. FIG. 17 is a diagram for explaining data distribution calculation processing according to the first embodiment. FIG. 18A is a diagram for explaining target point mark display processing in correction gazing point detection processing (step S102). FIG. 18-2 is a diagram for explaining target point mark display processing in the correction gazing point detection processing (step S102). FIG. 18C is a diagram for explaining the target point mark display process in the correction gazing point detection process (step S102). FIG. 18D is a diagram for explaining target point mark display processing in the correction gazing point detection processing (step S102). FIG. 18-5 is a diagram for explaining target point mark display processing in the correction gazing point detection processing (step S102). FIG. 19 is a flowchart showing a process of specifying a partial area in the frame Fe in the partial area specifying process (step S104) of the first embodiment. FIG. 20 is a flowchart illustrating a second diagnosis process performed by the diagnosis support apparatus according to the first embodiment. FIG. 21 is a flowchart showing detailed processing in the deficient data acquisition processing (step S204). FIG. 22-1 is a diagram for explaining target point mark display processing in the deficient data acquisition processing. FIG. 22-2 is a diagram for explaining target point landmark display processing in the deficient data acquisition processing. FIG. 22C is a diagram for explaining target point mark display processing in the deficient data acquisition processing. FIG. 22D is a diagram for explaining the target point mark display process in the deficient data acquisition process. FIG. 22-5 is a diagram for explaining the target point mark display process in the deficient data acquisition process. FIG. 23 is a functional block diagram of the diagnosis support apparatus according to the second embodiment. FIG. 24 is a diagram illustrating the positions of the correction gazing point data SPm0 to SPm4 when all the correction gazing points are detected. FIG. 25 is a diagram illustrating an example of prediction when the correction gazing point SPm1 cannot be detected. FIG. 26 is a diagram illustrating an example of prediction when the correction gazing point SPm4 cannot be detected. FIG. 27 is a diagram illustrating an example of prediction when the correction gazing points SPm2 and SPm4 cannot be detected. FIG. 28 is a diagram illustrating an example of prediction when correction gazing points SPm1 and SPm2 cannot be detected. FIG. 29 is a diagram illustrating an example of prediction when the correction gazing points SPm1, SPm2, and SPm3 cannot be detected. FIG. 30 is a flowchart illustrating a procedure of correction gazing point detection processing according to the second embodiment. FIG. 31 is a flowchart illustrating a procedure of correction gazing point prediction processing according to the second embodiment.

  Hereinafter, embodiments of a gaze point detection device, a gaze point detection method, a diagnosis support device, and a diagnosis support method will be described in detail with reference to the accompanying drawings.

(Embodiment 1)
FIG. 1 is a diagram illustrating a schematic configuration of a diagnosis support apparatus 100 including a gazing point detection apparatus according to the first embodiment. The diagnosis support apparatus 100 according to the present embodiment is an apparatus that performs a test for obtaining an evaluation value that is an index for diagnosis of developmental disorder by a doctor, that is, a test for supporting a diagnosis.

  One characteristic of children with developmental disabilities is that they do not see the eyes of the person they are facing. The diagnosis support apparatus 100 according to the present embodiment uses this feature, and when the face image is shown to the person to be inspected, that is, the examinee, the examinee looks at the eyes in the face image. Time and time other than eyes are measured, and an evaluation value is calculated based on the measurement result. Here, the inspected person is a to-be-detected person who is a target for gaze point detection.

  The diagnosis support apparatus 100 includes a first display unit 101, a second display unit 102, a speaker 103, a stereo camera 110, infrared LED light sources 121 and 122, a drive / IF unit 130, a control unit 140, And a storage unit 150.

  The first display unit 101 displays an image or the like that causes the subject to gaze. The second display unit 102 displays the operation content of the diagnosis support apparatus 100, the diagnosis result, and the like. The first display unit 101 and the second display unit 102 are, for example, monitor screens. The speaker 103 outputs a diagnosis result or the like as a sound. In addition, the speaker 103 outputs a sound for urging the person to be inspected during the inspection.

  The stereo camera 110 is installed below the first display unit 101 when viewed from the subject. The stereo camera 110 is an imaging unit that can perform stereo shooting with infrared rays, and includes a pair of cameras, a right camera 111 and a left camera 112. Hereinafter, the right camera 111 and the left camera 112 are referred to as left and right cameras 111 and 112, as appropriate.

  Around each lens of the right camera 111 and the left camera 112, infrared LED (Light Emitting Diode) light sources 121 and 122 are respectively arranged in the circumferential direction. The infrared LED light sources 121 and 122 include an inner peripheral LED and an outer peripheral LED. The inner and outer peripheral LEDs will be described later.

  Stereo camera 110 images the face of the subject. The stereo camera 110 captures an image at a timing when infrared rays are emitted from the infrared LED light sources 121 and 122. That is, the stereo camera 110 captures an image of infrared reflected light.

  The stereo camera 110 only needs to be installed at a position where the face of the subject can be imaged, and the installation position is not limited to the embodiment. Further, the image to be captured (hereinafter referred to as a captured image) may be an image including the pupil of the examinee, may be an image including only a part of the face including the pupil, and may be added to the face. It may be an image including other than the face.

  The drive / IF unit 130 drives each unit included in the stereo camera 110. The drive / IF unit 130 functions as an interface between each unit included in the stereo camera 110 and the control unit 140.

  The control unit 140 is connected to the first display unit 101, the second display unit 102, and the drive / IF unit 130. The control unit 140 controls images to be displayed on the first display unit 101 and the second display unit 102.

  The control unit 140 also detects the gaze point of the subject based on the captured image obtained by the stereo camera 110 and calibrates the position of the detected gaze point. The control unit 140 calculates an evaluation value for diagnosis of developmental disorder based on the detected gazing point.

  The storage unit 150 stores various information such as a control program, a measurement result, and a diagnosis support result. For example, the storage unit 150 stores an image to be displayed on the first display unit 101.

  FIG. 2 is a diagram for explaining a coordinate system used for processing of the diagnosis support apparatus 100. The first display unit 101 according to the present embodiment is a rectangular screen having a long side in the horizontal direction. As shown in FIG. 2, the diagnosis support apparatus 100 uses the center position of the screen of the first display unit 101 as an origin, the Y coordinate (upward is the + direction), and the horizontal coordinate X (the first display unit 101). The process is performed in an XYZ coordinate system in which the right is the + direction and the depth is the Z coordinate (the front of the first display unit 101 is the + direction).

  FIG. 3 is a block diagram illustrating an example of detailed functions of each unit illustrated in FIG. 1. The infrared LED light source 121 includes a wavelength 1-LED 123 that is an inner peripheral LED and a wavelength 2-LED 124 that is an outer peripheral LED. The infrared LED light source 122 includes a wavelength 1-LED 125 that is an inner peripheral LED and a wavelength 2-LED 126 that is an outer peripheral LED.

  Wavelength 1-LEDs 123 and 125 emit infrared light having a wavelength 1. The wavelength 2-LEDs 124 and 126 emit infrared light having a wavelength 2. Wavelength 1 and wavelength 2 are different from each other. Wavelength 1 and wavelength 2 are, for example, a wavelength of less than 900 nm and a wavelength of 900 nm or more, respectively. When the reflected light reflected by the pupil is irradiated with infrared light having a wavelength of less than 900 nm, a bright pupil image is obtained as compared with the case where reflected light reflected by the pupil is irradiated with infrared light having a wavelength of 900 nm or more. It is because it is obtained.

  The drive / IF unit 130 includes camera IFs 131 and 132, an LED drive control unit 133, and a speaker drive unit 134. The camera IF 131 and the camera IF 132 are connected to the right camera 111 and the left camera 112, respectively. The camera IFs 131 and 132 drive the left and right cameras 111 and 112. The right camera 111 transmits a frame synchronization signal to the left camera 112 and the LED drive control unit 133.

  The LED drive control unit 133 controls light emission of the wavelength 1-LEDs 123 and 125 and the wavelength 2 -LEDs 124 and 126. Specifically, the LED drive control unit 133 causes the left and right wavelength 1 infrared light sources (wavelength 1-LED 123, wavelength 1-LED 125) to emit light in the first frame at different timings based on the frame synchronization signal. . Corresponding to this light emission timing, the left and right cameras 111 and 112 capture images. The captured image is input to the control unit 140 via the camera IFs 131 and 132.

  Similarly, the LED drive control unit 133 causes the left and right wavelength 2 infrared light sources (wavelength 2-LED 124, wavelength 2-LED 126) to emit light at different timings in the second frame. Corresponding to this light emission timing, the left and right cameras 111 and 112 capture images. The captured image is input to the control unit 140 via the camera IFs 131 and 132.

  The speaker driving unit 134 drives the speaker 103. The control unit 140 controls the entire diagnosis support apparatus 100. Specifically, the control unit 140 includes a display control unit 141, a gazing point detection unit 142, an area dividing unit 143, an area specifying unit 144, a gazing point correction unit 145, and an evaluation unit 146. .

  The display control unit 141 controls display of various types of information on the first display unit 101 and the second display unit 102. The display control unit 141 displays an inspection image on the first display unit 101, for example. The inspection image is a target image for detecting the gaze point of the person to be inspected, and is specifically a human face image. More preferably, the inspection image is a face image of the mother of the subject. Here, the screen of the first display unit 101 is a display area for displaying an inspection image, and is a target area where a gazing point is detected.

  Further, the display control unit 141 displays a mark of the target gazing point position (target point) used for the gazing point position correction process on the first display unit 101. Here, the target point is a position to be watched by the subject.

  FIG. 4 is a diagram illustrating an example of a target point mark. As shown in FIG. 4, the mark is an image of a circle having a predetermined radius. In the present embodiment, the display control unit 141 displays five marks (SP0 to SP4) indicating five different target points in the display area on the first display unit 101.

  In the present embodiment, the quadrangle vertex positions having four sides parallel to each side of the rectangular screen (display area) are set as the target points of SP1 to SP4. Also, the approximate center of the screen is the SP0 target point. That is, SP0 is a center mark indicating an inner position in the display area as compared with SP1 to SP4. In the present embodiment, SP0 is assumed to coincide with the origin of the XYZ coordinate system, that is, the center position of the screen of the first display unit 101. It is assumed that the type and display position of the target point mark image are set in advance in the storage unit 150 or the like.

  As another example, the display control unit 141 may perform display control such as changing the radius of a circle image as a mark within a predetermined range. Thereby, it is possible to easily direct the attention of the subject to the target point.

  The mark of the target point may be an image that is different in luminance and saturation from other regions of the first display unit 101, and may be an image other than a circular image. As another example, the target point mark may be indicated by irradiating light to the target point arrangement position. That is, the mark may be light other than the image. As described above, the target point only needs to be an aspect that allows the subject to recognize the position to be watched, and the display aspect is not limited to the embodiment.

  Returning to FIG. 3, the gazing point detection unit 142 detects the gazing point of the subject based on the image captured by the stereo camera 110. Specifically, the gaze point detection unit 142 detects the gaze point of the inspected person (hereinafter referred to as a correction gaze point) at the time of displaying the mark of the target point displayed for the above correction. Note that the gazing point detection unit 142 calculates an average of a plurality of gazing point positions obtained for the same mark, and sets this as the gazing point position. The gaze point detection unit 142 also detects a gaze point of the inspected person (hereinafter referred to as an examination gaze point) when the examination image is displayed. The inspection gazing point is a target gazing point to be detected by the diagnosis support apparatus 100 including the gazing point detection apparatus.

  As a gazing point detection method by the gazing point detection unit 142, any conventionally used method can be applied. Below, the case where a test subject's viewpoint is detected using a stereo camera is demonstrated to an example similarly to the technique described in Unexamined-Japanese-Patent No. 2005-198743.

  The gazing point detection unit 142 detects the line-of-sight direction of the subject from the image captured by the stereo camera 110. The gaze point detection unit 142 detects the line-of-sight direction of the subject using, for example, the methods described in Japanese Patent Application Laid-Open Nos. 2011-206542 and 2008-125629. Specifically, the gazing point detection unit 142 obtains a difference between an image captured by irradiating infrared light having a wavelength 1 and an image captured by irradiating infrared light having a wavelength 2, and an image in which a pupil image is clarified. Is generated. The gazing point detection unit 142 calculates the position of the pupil of the subject using a stereo vision technique using the two images generated as described above from the images captured by the left and right cameras 111 and 112, respectively. In addition, the gazing point detection unit 142 calculates the position of the corneal reflection of the subject using the images captured by the left and right cameras 111 and 112. Then, the left and right cameras 111 and 112 calculate a line-of-sight vector that represents the line-of-sight direction of the subject from the position of the pupil of the subject and the corneal reflection position.

  The gazing point detection unit 142 detects, for example, the intersection of the line-of-sight vector represented in the coordinate system as shown in FIG. 2 and the XY plane as the gazing point of the subject. When the gaze direction of both eyes is obtained, the gaze point may be measured by obtaining the intersection of the left and right gazes of the subject.

  Note that the method of detecting the gaze point of the subject is not limited to this. For example, the gaze point of the person to be inspected may be detected by analyzing an image captured using visible light instead of infrared light.

  FIG. 5 is an explanatory diagram schematically showing a method for detecting the position of the pupil of the subject. The gaze point detection unit 142 calculates the direction of the line of sight of the subject 201 viewed from the right camera 111 from the position of the pupil detected by the right camera 111 and the positional relationship of corneal reflection. The gazing point detection unit 142 calculates the direction of the line of sight of the subject 201 viewed from the left camera 112 from the position of the pupil detected by the left camera 112 and the positional relationship of corneal reflection. Then, the gazing point detection unit 142 calculates the position of the gazing point of the inspected person 201 in the XY coordinate system illustrated in FIG. 2 from the intersection of the obtained two line-of-sight directions.

  Returning to FIG. 3, the area dividing unit 143 divides the screen of the first display unit 101 into a plurality of partial areas based on the correction gazing point detected with respect to the target point mark displayed for correction. .

  The area specifying unit 144 specifies which partial area in the display area the inspection gazing point detected for the inspection image belongs to.

  The gazing point correction unit 145 corrects the position of the inspection gazing point based on the partial region to which the inspection gazing point belongs.

  Here, the processing of the region dividing unit 143, the region specifying unit 144, and the gaze point correcting unit 145 will be described in detail with reference to FIGS.

  Points SP0a to SP4a shown in FIG. 6 are correction point of sight of the inspected person obtained by the inspected person gazing at the marks SP0 to SP4 of the target points, respectively.

  The area dividing unit 143 first matches the point SP0a with the origin of the XY coordinate system. Specifically, the region dividing unit 143 calculates an origin displacement value from the origin of the XY coordinate system to SP0a. Here, the origin and the position of SP0 coincide. Then, the region dividing unit 143 obtains the coordinates of SP1b to SP4b obtained by moving each of SP1a to SP4a by the origin displacement value. For example, when the coordinates of SP0a are (x0, y0), the origin displacement value is (−x0, −y0). Therefore, the area dividing unit 143 obtains the coordinates of SP1b to SP4b by adding (−x0, −y0) to the coordinates of SP1a to SP4a.

  FIG. 7 is a diagram showing the frame F and the frame Fe after the point SP0a in FIG. 6 is moved to the origin of the XY coordinate system. The frame F is a frame determined by the target point points F1 to F4. Points F1 to F4 correspond to points SP1 to SP4, respectively. The frame Fe is a frame determined by points Fe1 to Fe4 corresponding to the correction gazing points detected corresponding to the target points. Points Fe1 to Fe4 correspond to points SP1b to SP4b, respectively. That is, the frame F is a reference frame corresponding to the XY coordinate system of the display screen, and the frame Fe is an error frame including an error unique to the subject. Note that the frame F and the frame Fe have different concepts from the frames to be imaged by the left and right cameras 111 and 112.

  The inspection gazing point calculated by the gazing point detection unit 142 is a point on the frame Fe. Furthermore, due to the aspherical shape of the eyeball of the examinee, the degree of error varies depending on the line-of-sight direction, that is, the mark position. Therefore, in the diagnosis support apparatus 100 according to the present embodiment, the target area is divided into eight partial areas, and the coordinate values on the frame Fe of the gazing point for inspection are converted into the coordinate values on the frame F in units of partial areas. By doing so, the gaze point for inspection will be corrected.

  The area dividing unit 143 divides the target area into eight partial areas based on the positions of the points Fe1 to Fe4. FIG. 8 is a diagram showing eight partial areas. Each partial region is a region determined by the origin O, points Fe1 to Fe4, the X axis, and the Y axis. Here, the points Fe1 to Fe4 are points corresponding to SP1b to SP4b, respectively. SP1b to SP4b are points determined based on SP1a to SP4a, respectively. That is, the region dividing unit 143 divides the target region into eight partial regions based on the positions of SP1a to SP4a.

  Here, the intersection of the straight line passing through the points Fe1 and Fe4 and the X axis is defined as a point Fe5. An intersection point between a straight line passing through the points Fe1 and Fe2 and the Y axis is defined as a point Fe6. A point of intersection between the straight line passing through the points Fe2 and Fe3 and the X axis is defined as a point Fe7. A point of intersection between the straight line passing through the points Fe3 and Fe4 and the Y axis is defined as a point Fe8.

  The area dividing unit 143 includes a first quadrant lower area A, a first quadrant upper area B, a second quadrant upper area C, a second quadrant lower area D, a third quadrant upper area E, and a third quadrant shown in FIG. The area is divided into eight partial areas of a lower area F, a fourth quadrant lower area G, and a fourth quadrant upper area H. In addition, the region specifying unit 144 determines whether the inspection point of interest detected by the point of interest detection unit 142 is based on the coordinates of the inspection point of interest and the positions of the eight partial regions divided by the region dividing unit 143. Which of the eight partial areas belongs to which partial area is specified.

  Here, the lower quadrant A of the first quadrant is an area in which a straight line passing through the origin O and the point Fe1 and a straight line passing through the origin O and the point Fe5 (X axis) are boundary positions. The first quadrant upper area B is an area having a boundary position that is a straight line passing through the origin O and the point Fe1 and a straight line passing through the origin O and the point Fe6 (Y axis).

  The second quadrant upper area C is an area having a straight line passing through the origin O and the point Fe2 and a straight line passing through the origin O and the point Fe6 (Y axis) as boundary positions. The lower quadrant area D is an area having a boundary position that is a straight line passing through the origin O and the point Fe2 and a straight line passing through the origin O and the point Fe7 (X axis).

  The third quadrant upper area E is an area having a straight line passing through the origin O and the point Fe3 and a straight line passing through the origin O and the point Fe7 (X axis) as boundary positions. The lower quadrant F of the third quadrant is an area having a boundary position that is a straight line passing through the origin O and the point Fe3 and a straight line passing through the origin O and the point Fe8 (Y axis).

  The lower quadrant region G is a region having a boundary position that is a straight line passing through the origin O and the point Fe4 and a straight line passing through the origin O and the point Fe8 (Y axis). The upper quadrant H of the fourth quadrant is an area having a straight line passing through the origin O and the point Fe4 and a straight line passing through the origin O and the point Fe5 (X axis) as boundary positions.

  FIG. 9 shows a first quadrant lower area A, a first quadrant upper area B, a second quadrant upper area C, a second quadrant lower area D, a third quadrant upper area E, a third quadrant lower area F, It is a figure which shows the partial area | region (A'-H ') of the flame | frame F corresponding to 4th quadrant lower area | region G and 4th quadrant upper area | region H, respectively.

  First quadrant lower area A ′, first quadrant upper area B ′, second quadrant upper area C ′, second quadrant lower area D ′, third quadrant upper area E ′, third quadrant lower area F ′ , The fourth quadrant lower region G ′ and the fourth quadrant upper region H ′ are respectively a first quadrant lower region A, a first quadrant upper region B, a second quadrant upper region C, a second quadrant lower region D, This is a region corresponding to the third quadrant upper region E, the third quadrant lower region F, the fourth quadrant lower region G, and the fourth quadrant upper region H.

  The area specifying unit 144 specifies a partial area on the frame F corresponding to the specified partial area on the frame Fe, in addition to specifying the partial area on the frame Fe to which the inspection point of interest belongs as described above. To do. For example, when the first quadrant lower area A on the frame Fe is specified, the first quadrant lower area A ′ on the frame F is specified as the corresponding partial area. That is, the region specifying unit 144 specifies a partial region on the frame Fe and a corresponding partial region on the frame F, and corresponds to a first partial region specifying unit and a second partial region specifying unit. It is.

  The partial area on the frame F will be described with reference to FIG. A point of intersection between the straight line passing through the points F1 and F4 and the X axis is defined as a point F5. A point of intersection between the straight line passing through the points F1 and F2 and the Y axis is defined as a point F6. A point of intersection between the straight line passing through the points F2 and F3 and the X axis is defined as a point F7. A point of intersection between the straight line passing through the points F3 and F4 and the Y axis is defined as a point F8.

  The first quadrant lower area A 'is an area having a boundary position that is a straight line passing through the origin O and the point F1 and a straight line passing through the origin O and the point F5 (X axis). The first quadrant upper area B 'is an area having a boundary position that is a straight line passing through the origin O and the point F1 and a straight line passing through the origin O and the point F6 (Y axis).

  The second quadrant upper area C 'is an area having a boundary position that is a straight line passing through the origin O and the point F2 and a straight line passing through the origin O and the point F6 (Y axis). The second quadrant lower area D 'is an area having a boundary position that is a straight line passing through the origin O and the point F2 and a straight line passing through the origin O and the point F7 (X axis).

  The third quadrant upper area E 'is an area having a boundary position that is a straight line passing through the origin O and the point F3 and a straight line passing through the origin O and the point F7 (X axis). The third quadrant lower area F ′ is an area having a boundary position that is a straight line passing through the origin O and the point F3 and a straight line passing through the origin O and the point F8 (Y axis).

  The fourth quadrant lower region G ′ is a region having a boundary position that is a straight line passing through the origin O and the point F4 and a straight line passing through the origin O and the point F8 (Y axis). The upper quadrant H ′ of the fourth quadrant is an area having a boundary position that is a straight line passing through the origin O and the point F4 and a straight line passing through the origin O and the point F5 (X axis).

  The gazing point correction unit 145 corrects the position of the gazing point based on the correlation between the partial area on the frame Fe and the corresponding partial area in the frame F, that is, the displacement value from the frame Fe to the frame F.

  Hereinafter, the process of the gaze point correction unit 145 will be described in detail with reference to FIGS. FIG. 10 is a diagram showing the coordinates of each point. As shown in FIG. 10, the coordinates of the point F1 are (x4, y4), the coordinates of the point F2 are (x10, y4), the coordinates of the point F3 are (x10, y8), and the coordinates of the point F4 are (x4, y8). The coordinates of the point F5 are (x4, 0), the coordinates of the point F6 are (0, y4), the coordinates of the point F7 are (x10, 0), and the coordinates of the point F8 are (0, y8). Also, the coordinates of the point Fe1 are (x6, y6), the coordinates of the point Fe2 are (x11, y11), the coordinates of the point Fe3 are (x13, y13), the coordinates of the point Fe4 are (x9, y9), and the coordinates of the point Fe5 Is (x7,0), the coordinates of the point Fe6 are (0, y7), the coordinates of the point Fe7 are (x20,0), and the coordinates of the point Fe8 are (0, y20).

  Further, as shown in FIG. 11, the inspection point of gaze detected in the lower quadrant A is a point P2A, and the coordinates of the point P2A are (x2A, y2A). Further, the point of gazing for inspection after the correction of the point P2A is the point P0A, and the coordinates of the point P0A are (x0A, y0A).

  Further, a straight line passing through the points Fe1 and Fe5 is L1A, and a straight line passing through the points F1 and F5 is L2A. A straight line passing through the point P2A and the origin O is L3A, a straight line passing through the point P0A and the origin is L4A, a straight line passing through the point Fe1 and the origin is L5A, and a straight line passing through the point F1 and the origin is L6A.

  Further, an intersection of the straight line L2A and the straight line L4A is a point P1A, and the coordinates of the point P1A are (x4, y1A). Further, an intersection of the straight line L1A and the straight line L3A is a point P3A, and the coordinates of the point P3A are (x3A, y3A).

  Further, the distance between the origin and the point P0A is d0A, the distance between the origin and the point P1A is d1A, the distance between the origin and the point P2A is d2A, and the distance between the origin and the point P3A is d3A.

  In addition, the angle 0A formed by the straight line L3A and the X axis is θ0A, the angle 1A formed by the straight line L5A and the X axis is θ1A, the angle 2A formed by the straight line L4A and the X axis is θ2A, and the straight line L6A and the X axis are The angle 3A formed is defined as θ3A.

  Under the above conditions, the gaze point correcting unit 145 assumes that the ratio of the angle 0A to the angle θ1A of the angle 1A and the angle θ1A of the angle 2A to the angle θ3A of the angle 2A is equal to the angle θ2A of the angle 2A. Further, assuming that the ratio of the distance d2A to the distance d3A is equal to the ratio of the distance d0A to the distance d1A, the position d0A is calculated to calculate the corrected position of the gazing point for correction.

First, the gaze point correcting unit 145 calculates the angle θ0A, the angle θ1A, and the angle θ3A using (Expression 1) to (Expression 3).

Since the ratio of the angle 0A to the angle θ1A of the angle 1A and the angle θ1A of the angle 1A is equal to the ratio of the angle θ2A of the angle 2A to the angle θ3A of the angle 3A, (Equation 4) holds. The gazing point correction unit 145 calculates the angle θ2A by substituting the angle θ0A, the angle θ1A, and the angle θ3A calculated by (Expression 1) to (Expression 3) into (Expression 4).

（式５）は、（式６）のように示すことができる。

（式６）において、ａｌおよびｂｌはそれぞれ（式７）および（式８）で示される。

また、直線Ｌ３Ａは、（式９）により示される。

（式９）を（式６）に代入することにより、（式１０）および（式１１）が得られる。

注視点補正部１４５は、（式１０）および（式１１）により、点Ｐ３Ａの座標（ｘ３Ａ，ｙ３Ａ）を算出する。 The gaze point correction unit 145 also calculates the coordinates (x3A, y3A) of the point P3A, assuming that the point 3A is an intersection of the straight line L1A and the straight line L3A. Here, the straight line L1A is represented by (Equation 5).

(Expression 5) can be expressed as (Expression 6).

In (Formula 6), al and bl are represented by (Formula 7) and (Formula 8), respectively.

The straight line L3A is represented by (Equation 9).

By substituting (Equation 9) into (Equation 6), (Equation 10) and (Equation 11) are obtained.

The gaze point correction unit 145 calculates the coordinates (x3A, y3A) of the point P3A by using (Expression 10) and (Expression 11).

The gaze point correcting unit 145 further calculates a ratio k0 of the distance d2A to the distance d3A by (Equation 12).

（式１３）を用いて、（式１４）および（式１５）が得られる。

以上に基づき、注視点補正部１４５は、（式１４）および（式１５）により、点Ｐ１Ａの座標（ｘ４，ｙ１Ａ）を算出する。 The gaze point correction unit 145 also calculates the coordinates (x4, y1A) of the point P1A. Here, the straight line L2A is represented by (Equation 13).

Using (Expression 13), (Expression 14) and (Expression 15) are obtained.

Based on the above, the gaze point correction unit 145 calculates the coordinates (x4, y1A) of the point P1A using (Expression 14) and (Expression 15).

The gaze point correcting unit 145 further calculates the distance d1A by (Equation 16).

The gazing point correction unit 145 calculates the coordinates (x0A, y0A) of the point P0A according to (Expression 17) and (Expression 18), assuming that the point P0A is a point where the ratio of the distance d0A to the distance d1A is k0. To do.

  Note that the gazing point correction unit 145 corrects the position of the inspection gazing point belonging to each partial area in the other partial areas using the same arithmetic expression.

  FIG. 12 is a diagram illustrating the upper quadrant H of the fourth quadrant. When the inspection gazing point belongs to the upper quadrant H of the fourth quadrant, the angles 0B to 3B are defined with reference to the X axis, and straight lines L1B to L6B are defined. Then, the gaze point correcting unit 145 obtains the angle θ2B of the angle 2B, assuming that the ratio of the angle 0B to the angle θ1B of the angle 1B is equal to the ratio of the angle 2B to the angle θ3B of the angle 2B. Further, the gaze point correction unit 145 determines the distance d0B on the assumption that the ratio of the distance d2B to the distance d3B is equal to the ratio of the distance d0B to the distance d1B. Thereby, the gazing point correction unit 145 calculates the coordinates (x0B, y0B) of the inspection gazing point P0B after correcting the inspection gazing point P2B (x2B, y2B).

  FIG. 13 is a diagram showing the upper quadrant area C of the second quadrant. When the inspection gazing point belongs to the second quadrant upper area C, the angles 0C to 3C are defined with reference to the Y axis, and straight lines L1C to L6C are defined. Then, the gazing point correction unit 145 obtains the angle θ2C of the angle 2C on the assumption that the ratio of the angle 0C to the angle θ1C of the angle 1C is equal to the ratio of the angle 2C of the angle 2C to the angle θ3C of the angle 3C. Furthermore, the gaze point correction unit 145 determines the distance d0C on the assumption that the ratio of the distance d2C to the distance d3C is equal to the ratio of the distance d0C to the distance d1C. Thereby, the gaze point correction unit 145 calculates the coordinates (x0C, y0C) of the inspection point of interest P0C after the correction of the inspection point of interest P2C (x2C, y2C).

  FIG. 14 is a diagram showing the first quadrant upper area B. As shown in FIG. When the inspection gazing point belongs to the first quadrant upper area B, the angles 0D to 3D are defined with reference to the Y axis, and the straight lines L1D to L6D are defined. Then, the gaze point correcting unit 145 obtains the angle θ2D of the angle 2D on the assumption that the ratio of the angle 0D to the angle θ1D of the angle 1D is equal to the ratio of the angle 2D of the angle 2D to the angle θ3D of the angle 3D. Further, the gaze point correction unit 145 determines the distance d0D on the assumption that the ratio of the distance d2D to the distance d3D is equal to the ratio of the distance d0D to the distance d1D. Thereby, the gazing point correction unit 145 calculates the coordinates (x0D, y0D) of the inspection gazing point P0D after correcting the inspection gazing point P2D (x2D, y2D).

  Returning to FIG. 3, the evaluation unit 146 calculates an evaluation value of the developmental disorder based on the inspection image and the corrected inspection point of interest. The evaluation unit 146 calculates, as an evaluation value, a ratio of looking at the eyes of the face image as the inspection image based on the position of the gaze point for inspection of the examinee when the inspection image is displayed, and the evaluation value is low The evaluation value that indicates that the possibility of developmental disorder is high is calculated. For details of the processing performed by the evaluation unit 146, reference can be made to, for example, JP 2011-206542A. The evaluation unit 146 may calculate the evaluation value based on the inspection image and the inspection gazing point, and the calculation method is not limited to the present embodiment.

  FIG. 15 is a flowchart showing the first diagnosis process by the diagnosis support apparatus 100. In the first diagnosis process, the diagnosis support apparatus 100 first executes simple calibration (step S100). In the simple calibration, the display control unit 141 displays one or two target points on the first display unit 101.

  Subsequently, the gazing point detection unit 142 detects a gazing point for calibration corresponding to the displayed target point. Then, the gazing point correction unit 145 performs simple gazing point calibration based on the detected calibration gazing point.

  In addition, when it is known that a very large shift of the gazing point does not occur, the simple calibration (step S100) may be omitted.

  Next, the diagnosis support apparatus 100 executes inspection gazing point detection processing (step S101). In the inspection gazing point detection process, first, the display control unit 141 displays an inspection image on the first display unit 101. Next, the gazing point detection unit 142 detects the gazing point of the inspected person when the inspection image is displayed on the first display unit 101, that is, the inspection gazing point.

  Next, the diagnosis support apparatus 100 executes correction gazing point detection processing (step S102). In the correction gazing point detection process, the diagnosis support apparatus 100 detects a gazing point with respect to the mark indicating the target point, that is, a correction gazing point. The correction gazing point detection process will be described later.

  Subsequently, the area dividing unit 143 divides the target area into eight partial areas (A to H) based on the correction gazing point (step S103). Next, the gazing point correction unit 145 identifies a partial region in the frame Fe to which the inspection gazing point obtained in step S101 belongs, and further identifies a partial region in the frame F corresponding to the identified partial region. (Step S104). Processing for specifying the partial area will be described later.

  Next, the gazing point correction unit 145 calibrates the position of the inspection gazing point (step S105). In the inspection gazing point calibration process in step S105, the gazing point detection unit 145 calculates a post-calibration position of the inspection gazing point by executing different calculation processes for each partial region identified in step S104.

  Next, the evaluation unit 146 calculates an evaluation value based on the corrected position of the gazing point for inspection (step S106). The evaluation value is output to the second display unit 102, for example. Thus, the diagnosis process is completed.

  In the first diagnosis process, the correction gazing point detection process (step S102) is performed after the inspection gazing point detection process (step S101). As another example, the correction gazing point detection process (step S102) is performed. After step S102), an inspection gazing point detection process (step S101) may be performed.

  Next, the diagnosis support apparatus 100 executes correction gazing point detection processing (step S102). FIG. 16 is a flowchart showing details of the correction gazing point detection processing according to the first embodiment. In the correction gazing point detection process, first, the display control unit 141 displays the mark SP0 indicating the target point at a predetermined position (origin position) (step S110). Next, the gazing point detection unit 142 detects the position of the correction gazing point obtained for the mark SP0 (step S111). The gazing point detection unit 142 detects a predetermined number of correction gazing points and calculates the data distribution thereof (step S112).

  FIG. 17 is a diagram for explaining data distribution calculation processing. A reference area centered on the marks (SP0 to SP4) is set in advance for each mark. In the present embodiment, a circle having a predetermined radius centered on the mark is used as the reference region. If a predetermined number of calibration gaze points cannot be obtained within the reference area, it is likely that the inspector did not gaze closely at the target point mark, and a correction gaze point that is effective for correction is detected. It's likely not. Therefore, in the present embodiment, the gazing point detection unit 142 detects the correction point detected depending on whether or not a predetermined number or more of gazing points in the reference area are detected during a predetermined time while the landmark is displayed. Judgment is made on whether or not the gaze point is appropriate.

  In the data distribution calculation process in step S112, the gazing point detection unit 142 measures the number of correction gazing points belonging to the reference area. Then, the gazing point detection unit 142 determines that the detected gazing point for correction is appropriate when the number of gazing points for correction belonging to the reference area is equal to or greater than a preset threshold (step S113, Yes), the reference area data group excluding the correction gazing point that does not belong to the reference area among the detected calibration gazing points is specified (step S114).

  If it is determined in step S113 that the correction gazing point is not appropriate (No in step S113), the process returns to step S111, and the gazing point detection unit 142 again detects the correction gazing point.

  Next, the gazing point detection unit 142 calculates a representative value of the data group in the reference area based on the coordinates of each correction gazing point in the data group in the reference area (step S115). In the present embodiment, the average value of the data group in the reference area is calculated as the representative value. The representative value may be a value other than the average value. The representative value may be a standard deviation, for example. In the subsequent processing, the representative value calculated in step S115 is used as the correction gazing point.

  Next, when the representative values for the marks of all target points have not been calculated (No at Step S116), the display control unit 141 displays a mark indicating the next target point (Step S117). Then, the process returns to step S111, and representative values for the displayed mark are calculated (steps S111 to S115).

  In step S116, when representative values for all target point markers are calculated (step S116, Yes), the correction gazing point detection processing (step S102) is completed.

  FIG. 18A to FIG. 18E are diagrams for explaining the mark point display process of each target point in the correction gazing point detection process (step S102). In step S110 shown in FIG. 16, the display control unit 141 first displays the mark SP0 on the first display unit 101 as shown in FIG. Next, in step S117, the display control unit 141 displays the remaining target point marks one by one on the first display unit 101 in order.

  That is, the display control unit 141 stops displaying SP0 in step S117 as shown in FIG. 18-2, newly displays SP1, and again after calculating the representative value of SP1, in step S117, the display control unit 141 again displays in FIG. 18-3. Thus, the display of SP1 is stopped and SP2 is newly displayed. Similarly, the display control unit 141 continues to stop displaying SP2 and newly display the target point of SP3 as shown in FIG. 18-4. Subsequently, as shown in FIG. 18-5, the display of SP3 is stopped and the target point of SP4 is newly displayed.

  Thus, by displaying each target point one by one in order, it is possible to detect the gaze point for correction of the subject.

  FIG. 19 is a flowchart showing a process of specifying a partial area in the frame Fe in the partial area specifying process (step S104). As shown in FIG. 19, first, the region specifying unit 144 acquires the coordinates of the inspection gazing point obtained in step S101 from the gazing point detection unit 142 (step S120).

  Next, the area specifying unit 144 specifies the partial area to which the inspection gazing point belongs based on the coordinates of the inspection gazing point and the partial areas (A to H) on the frame Fe. Specifically, the area specifying unit 144 specifies the code of the x coordinate of the inspection gazing point, and then specifies the code of the y coordinate. When the x coordinate of the inspection gazing point is 0 or more and the y coordinate of the inspection gazing point is 0 or more (step S121, Yes, step S122, Yes), the region specifying unit 144 further performs inspection. Based on the xy coordinates of the gazing point and the coordinates of the point Fe1, it is determined whether or not the inspection gazing point is located above a straight line (straight line O-Fe1) connecting the origin O and the point Fe1.

  When the inspection gazing point is located on the straight line O-Fe1 or below the straight line O-Fe1 (No in step S123), the region specifying unit 144 sets the inspection gazing point in the first quadrant lower region A. It is determined that it belongs (step S124).

  When the inspection gazing point is located above the straight line O-Fe1 (step S123, Yes), the region specifying unit 144 determines that the inspection gazing point belongs to the first quadrant upper region B (step S125). .

  When the x coordinate of the inspection point of interest is 0 or more and the y coordinate of the inspection point of interest is smaller than 0 (step S121, Yes, step S122, No), the region specifying unit 144 further performs the inspection note. Based on the xy coordinates of the viewpoint and the coordinates of the point Fe4, it is determined whether or not the inspection gazing point is located above a straight line (straight line O-Fe4) connecting the origin O and the point Fe4.

  When the inspection gazing point is located on the straight line O-Fe4 or below the straight line O-Fe4 (No in step S126), the region specifying unit 144 sets the inspection gazing point in the lower quadrant G of the fourth quadrant. It is determined that it belongs (step S127).

  When the inspection gazing point is located above the straight line O-Fe4 (step S126, Yes), the region specifying unit 144 determines that the inspection gazing point belongs to the fourth quadrant upper region H (step S128). .

  When the x coordinate of the inspection gazing point is smaller than 0 and the y coordinate of the inspection gazing point is 0 or more (step S121, No, step S129, Yes), the region specifying unit 144 further performs the inspection note. Based on the xy coordinates of the viewpoint and the coordinates of the point Fe2, it is specified whether or not the inspection gazing point is located above the straight line connecting the origin O and the point Fe2 (straight line O-Fe2).

  When the inspection gazing point is located on the straight line O-Fe2 or below the straight line O-Fe2 (step S130, No), the region specifying unit 144 sets the inspection gazing point in the lower quadrant D of the second quadrant. It is determined that it belongs (step S131).

  When the inspection gazing point is located above the straight line O-Fe2 (step S130, Yes), the region specifying unit 144 determines that the inspection gazing point belongs to the second quadrant upper region C (step S132). .

  When the x coordinate of the inspection gazing point is smaller than 0 and the y coordinate of the inspection gazing point is smaller than 0 (step S121, No, step S129, No), the region specifying unit 144 further performs the inspection gazing point. Whether the inspection gazing point is located above the straight line connecting the origin O and the point Fe3 (straight line O-Fe3) is specified based on the xy coordinates of the point and the coordinates of the point Fe3.

  When the inspection gazing point is located on the straight line O-Fe3 or below the straight line O-Fe3 (step S133, No), the region specifying unit 144 sets the inspection gazing point in the lower quadrant F of the third quadrant. It is determined that it belongs (step S134).

  When the inspection gazing point is located above the straight line O-Fe3 (step S133, Yes), the region specifying unit 144 determines that the inspection gazing point belongs to the third quadrant upper region E (step S135). .

  This completes the process of specifying the partial region to which the inspection point of interest belongs in step S104.

  As described above, in the first diagnosis process, it is possible to accurately correct the gazing point position from only the correction gazing points corresponding to the five marks. That is, even after simple calibration is performed in a short time, the correction can be satisfactorily performed.

  FIG. 20 is a flowchart illustrating the second diagnosis process performed by the diagnosis support apparatus 100. In the second diagnostic process, a series of processes for detecting the gazing point for inspection are executed in three steps, diagnostic measurement 1, diagnostic measurement 2 and diagnostic measurement 3, and each process ends and the next process starts. Processing for detecting a gazing point for correction is performed before each step, that is, between each step. Thus, in the second diagnosis process, the timings of the inspection gazing point detection process and the correction gazing point detection process are different from those of the first diagnosis process.

  Specifically, as shown in FIG. 20, the diagnosis support apparatus 100 first executes simple calibration as necessary (step S200). Next, the gazing point detection unit 142 sets “1” to the failure flag corresponding to each mark (step S201). The initial value of the failure flag is “1”, and the value is changed to “0” when a correction gazing point corresponding to each mark is obtained. The failure flag is stored in the storage unit 150, for example.

  Subsequently, the diagnosis support apparatus 100 executes the diagnostic measurement 1 for detecting the inspection gazing point (step S202). Subsequently, the gazing point detection unit 142 refers to the value of the failure flag corresponding to each mark. If there is a failure flag, that is, if there is a failure flag for which 1 is set (step S203, Yes), deficient data for the correction gazing point is acquired (step S204). On the other hand, when the failure flag does not exist (step S203, No), the process proceeds to step S205.

  FIG. 21 is a flowchart showing detailed processing in the deficient data acquisition processing (step S204). First, the gazing point detection unit 142 refers to the failure flag of SP0. If the failure flag of SP0 is 1 (step S220, Yes), the display control unit 141 displays the target point mark SP0 on the first display unit 101 (step S221). In step S220, when the failure flag of SP0 is not 1 (step S220, No), it progresses to step S228.

  The processing in steps S222 to S226 is the same as the processing in steps S111 to S115 described with reference to FIG. However, if it is determined in step S224 that it is not appropriate (No in step S224), the process proceeds to step S228 without returning to step S222.

  In addition, the gaze point detection unit 142 sets “0” to the failure flag corresponding to the mark to be processed, that is, the mark displayed on the first display unit 101 at this time, after the representative value calculation process (step S226). "Is set (step S227).

  Next, the gaze point detection unit 142 confirms whether or not an unprocessed target point exists. When there is an unprocessed target point, that is, when there is an SP to be processed next (step S228, Yes), the gazing point detection unit 142 refers to the value of the failure flag of the next SP. When the failure flag of the next SP is 1 (step S229, Yes), the display control unit 141 first displays the mark SP0 and moves and displays the mark to the next target point (step S230). Next, the display control unit 141 displays the mark SP of the next target point (step S231). Then, returning to step S222, the gaze point detection unit 142 detects a correction gaze point corresponding to the next mark.

  On the other hand, if the failure flag of the next SP is not 1 in step S229 (No in step S229), the process returns to step S228.

  In step S228, when there is no unprocessed target point, that is, when processing for all SPs is completed and there is no SP to be processed next (step S228, No), insufficient data acquisition processing ( Step S204) is completed.

  FIG. 22-1 to FIG. 22-5 are diagrams for explaining target point mark display processing in the deficient data acquisition processing (step S204). In step S221 shown in FIG. 21, the display control unit 141 first displays the SP0 mark on the first display unit 101 as shown in FIG. Next, in step S230, the display control unit 141 moves and displays the mark from SP0 to the next SP, and then displays the next mark on the first display unit 101 in order.

  That is, the display control unit 141 moves and displays the mark from SP0 to SP1 as shown in FIG. 22-2 in step S230, and then displays SP1 in step S231. Subsequently, the display control unit 141 moves and displays the mark from SP0 to SP2 as shown in FIG. 22-3 again in step S230 after the processing related to SP1, and then displays SP2 in step S231. Similarly, as shown in FIG. 22-4, the display control unit 141 moves and displays the mark from SP0 to SP3, and then displays SP3. Subsequently, as shown in FIG. 22-5, from SP0 to SP4. After the mark is moved and displayed, SP4 is displayed.

  Returning to FIG. 20, when the deficient data acquisition process (step S204) in step S204 is completed, the diagnosis support apparatus 100 next executes the diagnostic measurement 2 (step S205). Subsequently, when there is a failure flag (Yes at Step S206), the gazing point detection unit 142 acquires insufficient data (Step S207). If there is no failure flag (No at Step S206), the process proceeds to Step S208.

Next, the diagnosis support apparatus 100 executes diagnosis measurement 3 (step S208). continue,
If there is a failure flag (Yes at Step S209), the gazing point detection unit 142 acquires insufficient data (Step S210) and returns to Step S209. If the failure flag does not exist (step S209, No), the process proceeds to step S211, the gazing point is calibrated (step S211), and an evaluation value is calculated based on the gazing point after calibration (step S212). The second diagnosis process is completed.

  The insufficient data acquisition process (steps S207 and S210) is the same process as the insufficient data acquisition process (step S204).

  As described above, in the second diagnosis process, the correction gazing point detection process is performed between the diagnostic measurements, and in each correction gazing point detection process, correction is performed until an appropriate correction gazing point is detected. Instead of repeating the detection of the gazing point, the process is temporarily ended, and in the next correction gazing point detection process, the correction gazing point for which the representative value calculation has not been completed is detected. Can reduce stress on the test.

  As described above, in the evaluation value calculation process performed by the diagnosis support apparatus 100 according to the present embodiment, the timing for detecting the correction gazing point may be either before or after the inspection gazing point is detected. The gazing point may be detected.

  For this reason, even when the infant who is difficult to concentrate on the inspection for a long time is set as the person to be inspected, the position of the inspection gazing point can be accurately calibrated.

  As described above, in the diagnosis process performed by the diagnosis support apparatus 100 according to the present embodiment, it is possible to accurately correct the gazing point position from only the correction gazing points corresponding to the five marks. That is, it is possible to accurately correct the viewpoint position without simple calibration or calibration in a short time. Therefore, the measurement can be started immediately and is suitable for measurement of infants. The measurement for correction does not have any problem at any timing.

  As described above, the present invention has been described using the embodiment, but various changes or improvements can be added to the above embodiment.

  As such a first modification, in the present embodiment, the gazing point detection unit 142 detects calibration gazing points for five target points, but sets more than five target points, It is good also as detecting the gazing point for calibration with respect to each target point. In this case, the area dividing unit 143 divides the display area into more than eight partial areas based on these. The gazing point correction unit 145 calibrates the inspection gazing point by calculation corresponding to each partial region. Thereby, it is possible to calibrate the gazing point for inspection with higher accuracy.

  As a second modification, the display control unit 141 may not display the mark SP0 of the central target point serving as a reference. In this case, the region dividing unit 143 performs processing by setting the average value of the marks SP1 to SP4 as SP0 and the average value of SP1a to SP4a as SP0a.

  As a third modification, the frame Fe may be defined by points SP0a to SP4a instead of being defined by the origin and points SP1b to SP4b. In this case, an angle formed by a straight line passing through the point SP0a and parallel to the x-axis or the y-axis and a straight line passing through each of the points SP1a to SP4a and the point SP0a may be defined as an angle on the frame Fe used for calibration. . For example, in the first quadrant lower region shown in FIG. 11, an angle formed by a straight line passing through the point SP0a and parallel to the x axis and a straight line passing through the points SP0a and SP1a may be the angle θ1A.

(Embodiment 2)
In the first embodiment, when the correction gazing point for the mark necessary for correcting the gazing point position is not detected, the correction gazing point is detected again. However, in a one-and-a-half year medical checkup, the subject is an infant with weak concentration, and it is difficult to cause the screen to be watched many times in order to detect the correction gazing point again.

  For this reason, in this embodiment, when the correction gazing point cannot be detected, the correction characteristic that cannot be detected from the correction gazing point data that can be detected using the eye characteristics or the like is used. The position of the gazing point is predicted, and the predicted position of the gazing point for correction is used for correcting the position of the gazing point.

  FIG. 23 is a block diagram illustrating an example of detailed functions of each unit of the diagnosis support apparatus 2300 including the gazing point detection apparatus according to the second embodiment. As shown in FIG. 23, a diagnosis support apparatus 2300 according to the present embodiment includes a stereo camera 110, infrared LED light sources 121 and 122, a speaker 103, a drive / IF unit 130, a control unit 2340, and a storage unit. 150, a first display unit 101, and a second display unit 102 are mainly provided. Here, the functions and configurations of the stereo camera 110, the infrared LED light sources 121 and 122, the speaker 103, the drive / IF unit 130, the storage unit 150, the first display unit 101, and the second display unit 102 are described in the first embodiment. Since it is the same as that of FIG.

  The control unit 2340 is connected to the first display unit 101, the second display unit 102, and the drive / IF unit 130 as in the first embodiment. In addition, the control unit 2340 controls images to be displayed on the first display unit 101 and the second display unit 102 as in the first embodiment.

  As shown in FIG. 23, the control unit 2340 includes a display control unit 141, a gazing point detection unit 142, a gazing point prediction unit 2341, an area division unit 2343, an area specifying unit 144, and a gazing point correction unit 145. The evaluation unit 146 is mainly provided. Here, since the functions of the display control unit 141, the gazing point detection unit 142, the region specifying unit 144, the gazing point correction unit 145, and the evaluation unit 146 are the same as those in the first embodiment, the description thereof is omitted.

  The gaze point prediction unit 2341 uses the gaze point detection unit 142 to determine a correction gaze point corresponding to the center mark and a correction gaze point corresponding to at least one of the four or more marks outside the center mark. Detected based on the detected position of the correction gazing point and the characteristics of the subject's eye when no correction gazing point corresponding to some of the four or more points is detected. The position of the correction gazing point that was not performed is predicted.

  In the diagnosis support apparatus 2300 of the present embodiment in which the stereo camera 110 is installed below the first display unit 101 or the diagnosis support apparatus in which the stereo camera 110 is installed on the first display unit 101, the subject is Since the subject's cornea is taken from below or above, the upper and lower distortions are different when detecting the gaze point, and the measurement distortions are also different vertically, but the left and right are often symmetrical. . For this reason, when the gaze point prediction unit 2341 according to the present embodiment cannot detect the gaze point for correction corresponding to the mark of the target point for correcting the position of the gaze point, the eye characteristics of the subject Is used to predict the position of the correction gazing point that could not be detected.

  FIG. 24 is a diagram illustrating the positions of the correction gazing point data SPm0 to SPm4 when the correction gazing point corresponding to all the marks is detected. As shown in FIG. 24, the correction gazing points SPm1 to SPm4 are symmetrical with respect to a perpendicular line passing through the correction gazing point SPm0 corresponding to the center mark SP0. The correction gazing points SPm1 to SPm4 are vertically symmetric with respect to a horizontal line passing through the correction gazing point SPm0.

  On the other hand, the center mark SP0 often appears first, and is often the center position of the first display unit 101, so that it is rarely missed by the subject, and the correction gazing point SPm0 corresponding to the center mark SP0 is small. There are very few detection failures.

  For this reason, when the gaze point prediction unit 2341 according to the present embodiment cannot detect the gaze point for correction, the position of the gaze point for correction detected and the gaze point for correction corresponding to the center mark SP0. A position that is line-symmetric with respect to a perpendicular line or a horizontal line passing through SPm0 is obtained as a position based on eye characteristics, and this position is predicted as the position of a correction gazing point that has not been detected.

  Returning to FIG. 23, the region dividing unit 2343 detects the position of the correction gazing point detected with respect to the target point mark displayed for correction, and the position of the correction gazing point predicted by the gazing point prediction unit 2341. Based on each, the screen of the first display unit 101 is divided into a plurality of partial areas (first partial areas).

  Hereinafter, the details of the prediction of the position of the correction gazing point by the gazing point prediction unit 2341 will be described with a specific example. Hereinafter, the horizontal line passing through the correction gazing point SPm0 corresponding to the center mark is taken as the X axis, and the perpendicular line passing through the correction gazing point SPm0 is taken as the Y axis.

  FIG. 25 is a diagram illustrating an example of prediction when the correction gazing point SPm1 cannot be detected. In FIG. 25, it is assumed that the correction gazing point SPm0 corresponding to the center mark has been completely displaced to the origin of the XY coordinate system.

  First, the gaze point predicting unit 2341 prioritizes symmetry with respect to the Y axis, so the Y coordinate value is left as it is. The gaze point prediction unit 2341 then determines an X coordinate value at a position that is line-symmetric with respect to the Y axis passing through the correction gaze point SPm0 from the detected data of the correction gaze point SPm2. Then, the gaze point prediction unit 2341 predicts the X and Y coordinates of the position as the position of the correction gaze point SPm1 that could not be detected.

  FIG. 26 is a diagram illustrating an example of prediction when the correction gazing point SPm4 cannot be detected. As in the example of FIG. 25, the gaze point predicting unit 2341 determines, as the position of the correction gazing point SPm4 that cannot be detected, from the data of the correction gazing point SPm3 that can be detected, the position that is line symmetric with respect to the Y axis. Predict.

  FIG. 27 is a diagram illustrating an example of prediction when two correction gazing points SPm2 and SPm4 cannot be detected. In this case, similarly to the example of FIG. 25, the gazing point prediction unit 2341 uses a position that is line-symmetric with respect to the detected data of the correction gazing point SPm1 and the Y axis as the position of the correction gazing point SPm2. A predicted and detected position of the correction gazing point SPm3 and a position symmetrical with respect to the Y axis are predicted as the position of the correction gazing point SPm4 that could not be detected.

  FIG. 28 is a diagram illustrating an example of prediction when two correction gazing points SPm1 and SPm2 cannot be detected. In this example, since there is no data obtained using left-right symmetry, the gaze point prediction unit 2341 predicts the positions of the correction gaze points SPm1 and SPm2 from the vertical symmetry. That is, the gazing point prediction unit 2341 predicts the detected data of the correction gazing point SPm4 and the position line-symmetric with respect to the X axis as the position of the correction gazing point SPm1 that could not be detected, and the detected correction. A position that is line-symmetric with respect to the data of the gazing point SPm3 and the X axis is predicted as the position of the correction gazing point SPm2 that could not be detected.

  FIG. 29 is a diagram illustrating an example of prediction when three correction gazing points SPm1, SPm2, and SPm3 cannot be detected. First, the gaze point prediction unit 2341 uses left-right symmetry to predict the position of the correction gazing point SPm3 that could not be detected from the data of the correction gazing point SPm4 that could be detected and the position that is line-symmetric with respect to the Y axis. To do. As a result, the same state as in the example of FIG. 28 is obtained. For this reason, the gaze point prediction unit 2341 uses vertical symmetry, and the position of the correction gazing point SPm1 that could not be detected is detected as the position of the correction gazing point SPm4 that is detected and the line symmetric with respect to the X axis. A position that is line-symmetric with respect to the X-axis and the data of the correction gazing point SPm3 predicted previously is predicted as the position of the correction gazing point SPm2 that could not be detected.

  In each of the above examples, the gazing point prediction unit 2341 generates correction gazing point data after predicting the position of the gazing point for correction.

  As described above, when the correction gazing point SPm0 for the center mark and the other one sight mark have been detected, the correction based on the approximate gazing point position can be performed. In particular, in a developmental disorder diagnosis support apparatus or the like, the target object for analysis is relatively large, so that even this correction can be measured with no problem.

  Next, correction gaze point detection processing and correction gaze point prediction processing according to the present embodiment configured as described above will be described. The first diagnosis process of the present embodiment is the same as that of the first embodiment described with reference to FIG. FIG. 30 is a flowchart illustrating details of the correction gazing point detection processing according to the second embodiment. FIG. 30 is executed as the correction gazing point detection process in step S102 in the first diagnosis process shown in FIG.

  In the correction gazing point detection process, first, the gazing point detection unit 142 clears all the creation flags to 0 (step S3012). Here, the creation flag is a flag indicating whether or not the correction gazing point data has been detected and predicted for each mark and the correction gazing point data has been created. When the creation flag is “0”, When the correction gazing point data has not yet been created and the creation flag is “1”, it indicates that the correction gazing point data has been created.

  The creation flag exists corresponding to each mark, that is, each of the correction gazing points SPm0 to SPm4. In other words, the gazing point detection unit 142 detects the correction gazing points SPm0 to SPm4 and generates data for the correction gazing points SPm0 to SPm4, and then sets the creation flag for the generated correction gazing points SPm0 to SPm4 to “1”. To "". When the gaze point prediction unit 2341 predicts the positions of the correction gazing points SPm1 to SPm4 and generates data of the correction gazing points SPm1 to SPm4, the creation flag for the generated correction gazing points SPm0 to SPm4 is set to “1”. Set to.

  Next, the display control unit 141 displays the center mark SP0 indicating the target point at a predetermined position (origin position) (step S110). Next, the gazing point detection unit 142 detects the position of the correction gazing point obtained for the center mark SP0 (step S111). The gazing point detection unit 142 detects a predetermined number of correction gazing points and calculates the data distribution thereof (step S112). The data distribution calculation process is performed in the same manner as in the first embodiment.

  In the data distribution calculation process in step S112, the gazing point detection unit 142 measures the number of correction gazing points belonging to the reference area. The gazing point detection unit 142 determines that the detected gazing point for correction is appropriate when the number of gazing points for correction belonging to the reference area is equal to or greater than a preset threshold (step S3013). Yes), the reference area data group excluding the correction gazing point that does not belong to the reference area among the detected correction gazing points is specified (step S114).

  If it is determined in step S3013 that the correction gazing point is not appropriate (step S3013, No), the process proceeds to step S3015. That is, when the correction gazing point is not appropriate, the gazing point detection unit 142 does not re-detect the correction gazing point.

  Next, as in the first embodiment, the gazing point detection unit 142 calculates the representative value of the reference area data group based on the coordinates of each correction gazing point in the reference area data group (step S115). . Then, the gazing point detection unit 142 sets the correction gazing point creation flag to be created to “1” (step S3014).

  Next, it is determined whether or not the next target point mark exists (step S3015). If the next mark exists (step S3015, Yes), the display control unit 141 sets the next target point. The mark to show is displayed (step S117). Then, the process returns to step S111, and representative values for the displayed mark are calculated (steps S111 to S115).

  If it is determined in step S3015 that there is no next mark (step S3015, No), the gazing point detection unit 142 checks whether there is a correction gazing point that has not been detected (step S3016). If there is no correction gazing point that has not been detected (step S3016, No), the process ends.

  On the other hand, when there is a correction gazing point that has not been detected (step S3016, Yes), the gazing point prediction unit 2341 executes a correction gazing point prediction process (step S3017).

  FIG. 31 is a flowchart illustrating a procedure of correction gazing point prediction processing according to the second embodiment. First, the gaze point prediction unit 2341 determines whether there is data for the correction gaze point SPm0 corresponding to the center mark and data for one other correction gaze point (step S3103). Hereafter, the data of the correction gazing point SPmX is simply referred to as “SPmX correction gazing point data”.

  If either the SPm0 correction gazing point data or one other correction gazing point data does not exist (step S3103: No), the process returns to step S111 in FIG. 30 to detect the correction gazing point. Try again.

  On the other hand, if there is SPm0 correction gazing point data and one other correction gazing point data (step S3103: Yes), the gazing point prediction unit 2341 detects the correction gazing point SPm1 and corrects SPm1. It is determined whether the gazing point data is present (created) or whether the creation flag of SPm1 is 1 (step S3104).

  If SPm1 correction gazing point data exists or the SPm1 creation flag is 1 (Yes in step S3104), the SPm1 correction gazing point data is used, and the gazing point prediction unit 2341 The creation flag for SPm1 is set to 1 (step S3105).

  On the other hand, in step S3104, when there is no SPm1 correction gazing point data and the SPm1 creation flag is 0 (step S3104: No), the gazing point prediction unit 2341 applies to the SPm1 correction gazing point data. Judge whether there is left-right symmetry data. That is, the gazing point prediction unit 2341 detects whether or not the correction gazing point SPm2 that is symmetrical with respect to the SPm1 correction gazing point data is detected and the SPm2 correction gazing point data is present (created), or SPm2 generation It is determined whether or not the flag is 1 (step S3106).

  When the SPm2 correction gazing point data exists or the SPm2 creation flag is 1 (step S3106: Yes), the gazing point prediction unit 2341 uses the SPm2 correction gazing point data to perform SPm2 correction. A position that is line-symmetric with respect to the gazing point data for Y-axis is predicted as the position of the gazing point data for SPm1 correction. That is, the gaze point prediction unit 2341 uses the same Y coordinate as the SPm2 correction gaze point data, and uses the X coordinate as a coordinate line symmetrical with respect to the X coordinate and the Y axis of the SPm2 correction gaze point data. Viewpoint data is generated (step S3107). Then, the gaze point prediction unit 2341 sets the SPm1 creation flag to 1 (step S3108).

  On the other hand, in step S3106, when there is no SPm2 correction gazing point data and the SPm2 creation flag is 0 (step S3106: No), the gazing point prediction unit 2341 applies to the SPm1 correction gazing point data. Determine whether there is vertical symmetry data. That is, the gaze point prediction unit 2341 detects whether or not the correction gazing point SPm4 that is vertically symmetrical with respect to the SPm1 correction gazing point data is detected and the SPm4 correction gazing point data exists (created), or the creation of SPm4 It is determined whether or not the flag is 1 (step S3109).

  If SPm4 correction gazing point data exists or the SPm4 creation flag is 1 (step S3109: Yes), the gazing point prediction unit 2341 uses this SPm4 correction gazing point data to perform SPm4 correction. A position that is line-symmetric with respect to the gazing point data for use and the X axis is predicted as the position of the gazing point data for SPm1 correction. That is, the gazing point prediction unit 2341 uses the same X coordinate as the SPm4 correction gazing point data and uses the Y coordinate as a coordinate line symmetrical to the Y coordinate of the SPm4 correction gazing point data and the X axis. Viewpoint data is generated (step S3110). Then, the gaze point prediction unit 2341 sets the creation flag of SPm1 to 1 (step S3111). Then, the process proceeds to next step S3112.

  Also, in step S3109, when there is no SPm4 correction gazing point data and the SPm4 creation flag is 0 (step S3109: No), the process proceeds to step S3112.

  In the processing after step S3112, the SPm2 correction gazing point data, the SPm3 correction gazing point data, and the SPm4 correction gazing point data are also processed in the same manner as described above.

  That is, the gazing point prediction unit 2341 determines whether the correction gazing point SPm2 is detected and SPm2 correction gazing point data is present (created) or whether the SPm2 generation flag is 1 ( Step S3112).

  If SPm2 correction gazing point data exists or the SPm2 generation flag is 1 (step S3112: Yes), the SPm2 correction gazing point data is used, and the gazing point prediction unit 2341 The creation flag for SPm2 is set to 1 (step S3113).

  On the other hand, in step S3112, when there is no SPm2 correction gazing point data and the SPm2 creation flag is 0 (step S3112: No), the gazing point prediction unit 2341 applies to the SPm2 correction gazing point data. Judge whether there is left-right symmetry data. In other words, the gaze point prediction unit 2341 detects whether or not the correction gazing point SPm1 symmetrical to the SPm2 correction gazing point data is detected and the SPm1 correction gazing point data exists (is created) or the SPm1 is generated. It is determined whether or not the flag is 1 (step S3114).

  If the SPm1 correction gazing point data exists or the SPm1 creation flag is 1 (step S3114: Yes), the gazing point prediction unit 2341 uses the SPm1 correction gazing point data to perform SPm1 correction. A position that is line-symmetric with respect to the gazing point data for Y and the Y axis is predicted as the position of the gazing point data for SPm2 correction. That is, the gaze point prediction unit 2341 uses the same Y coordinate as the SPm1 correction gaze point data, and uses the X coordinate as a coordinate line symmetrical with respect to the X coordinate and the Y axis of the SPm1 correction gaze point data. Viewpoint data is generated (step S3115). Then, the gaze point prediction unit 2341 sets the SPm2 creation flag to 1 (step S3116).

  On the other hand, in step S3114, when there is no SPm1 correction gazing point data and the SPm1 creation flag is 0 (step S3114: No), the gazing point prediction unit 2341 applies to the SPm2 correction gazing point data. Determine whether there is vertical symmetry data. That is, the gaze point prediction unit 2341 detects whether or not the correction gazing point SPm3 that is vertically symmetrical with respect to the SPm2 correction gazing point data is detected and the SPm3 correction gazing point data is present (created) or the creation of SPm3 It is determined whether or not the flag is 1 (step S3117).

  When the SPm3 correction gazing point data exists or the SPm3 creation flag is 1 (Yes in step S3117), the gazing point prediction unit 2341 uses the SPm3 correction gazing point data to perform SPm3 correction. A position that is line-symmetric with respect to the gazing point data for use and the X axis is predicted as the position of the gazing point data for SPm2 correction. That is, the gazing point prediction unit 2341 uses the same X coordinate as the SPm3 correction gazing point data, and uses the Y coordinate as a coordinate line symmetrical to the Y coordinate of the SPm3 correction gazing point data and the X axis. Viewpoint data is generated (step S3118). Then, the gaze point prediction unit 2341 sets the SPm2 creation flag to 1 (step S3119). Then, the process proceeds to next step S3120.

  Also, in step S3117, when there is no SPm3 correction gazing point data and the SPm3 creation flag is 0 (step S3117: No), the process proceeds to step S3120.

  Next, in step S3120, the gazing point prediction unit 2341 detects whether the correction gazing point SPm3 is detected and whether SPm3 correction gazing point data is present (created) or whether the SPm3 generation flag is “1”. Is determined (step S3120).

  If SPm3 correction gazing point data exists or the SPm3 creation flag is 1 (step S3120: Yes), the SPm3 correction gazing point data is used, and the gazing point prediction unit 2341 The creation flag for SPm3 is set to 1 (step S3121).

  On the other hand, in step S3120, when there is no SPm3 correction gazing point data and the SPm3 creation flag is 0 (step S3120: No), the gazing point prediction unit 2341 applies to the SPm3 correction gazing point data. Judge whether there is left-right symmetry data. That is, the gazing point prediction unit 2341 detects whether or not the correction gazing point SPm4 that is symmetrical with respect to the SPm3 correction gazing point data is detected and the SPm4 correction gazing point data is present (created) or the SPm4 is generated. It is determined whether or not the flag is 1 (step S3122).

  When the SPm4 correction gazing point data exists or the SPm4 creation flag is 1 (step S3122: Yes), the gazing point prediction unit 2341 uses the SPm4 correction gazing point data to perform SPm4 correction. A position that is line-symmetric with respect to the gazing point data for Y-axis is predicted as the position of the gazing point data for SPm3 correction. That is, the gazing point prediction unit 2341 uses the same Y coordinate as the SPm4 correction gazing point data, and uses the X coordinate as a coordinate line symmetrical with respect to the X coordinate and the Y axis of the SPm4 correction gazing point data. Viewpoint data is generated (step S3123). Then, the gaze point prediction unit 2341 sets the SPm3 creation flag to 1 (step S3124).

  On the other hand, if there is no SPm4 correction gazing point data in step S3122 and the SPm4 creation flag is 0 (step S3122: No), the gazing point prediction unit 2341 applies to the SPm3 correction gazing point data. Determine whether there is vertical symmetry data. In other words, the gaze point prediction unit 2341 detects whether or not the correction gazing point SPm2 that is vertically symmetrical with respect to the SPm3 correction gazing point data is detected and the SPm2 correction gazing point data exists (created) or the creation of SPm2 It is determined whether or not the flag is 1 (step S3125).

  When the SPm2 correction gazing point data exists or the SPm2 creation flag is 1 (step S3125: Yes), the gazing point prediction unit 2341 uses the SPm2 correction gazing point data to perform SPm2 correction. A position that is line-symmetric with respect to the gazing point data for X and the X axis is predicted as the position of the gazing point data for SPm3 correction. That is, the gazing point prediction unit 2341 uses the same X coordinate as the SPm2 correction gazing point data, and uses the Y coordinate as a coordinate line symmetric with respect to the Y coordinate of the SPm2 correction gazing point data and the X axis. Viewpoint data is generated (step S3126). Then, the gaze point prediction unit 2341 sets the SPm3 creation flag to 1 (step S3127). Then, the process proceeds to next step S3128.

  Also, in step S3125, when there is no SPm2 correction gazing point data and the SPm2 creation flag is 0 (step S3125: No), the process proceeds to step S3128.

  Next, in step S3128, the gazing point prediction unit 2341 detects whether or not the correction gazing point SPm4 is detected and SPm4 correction gazing point data is present (created) or whether the SPm4 generation flag is “1”. Is determined (step S3128).

  If SPm4 correction gazing point data exists or the SPm4 creation flag is 1 (step S3128: Yes), the SPm4 correction gazing point data is used, and the gazing point prediction unit 2341 The creation flag for SPm4 is set to 1 (step S3129).

  On the other hand, in step S3128, when there is no SPm4 correction gazing point data and the SPm4 creation flag is 0 (step S3128: No), the gazing point prediction unit 2341 applies to the SPm4 correction gazing point data. Judge whether there is left-right symmetry data. That is, the gaze point prediction unit 2341 detects whether or not the correction gazing point SPm3 that is symmetrical with respect to the SPm4 correction gazing point data is detected and the SPm3 correction gazing point data exists (is created), or the SPm3 is generated. It is determined whether or not the flag is 1 (step S3130).

  If SPm3 correction gazing point data is present or the SPm3 creation flag is 1 (step S3130: Yes), the gazing point prediction unit 2341 uses this SPm3 correction gazing point data to perform SPm3 correction. A position that is line-symmetric with respect to the gazing point data for use and the Y axis is predicted as the position of the gazing point data for SPm4 correction. That is, the gaze point prediction unit 2341 uses the same Y coordinate as the SPm3 correction gaze point data, and uses the X coordinate as a coordinate line symmetrical with respect to the X coordinate and the Y axis of the SPm3 correction gaze point data. Viewpoint data is generated (step S3131). Then, the gaze point prediction unit 2341 sets the creation flag of SPm4 to 1 (step S3132).

  On the other hand, in step S3130, when there is no SPm3 correction gazing point data and the SPm3 creation flag is 0 (step S3130: No), the gazing point prediction unit 2341 applies to the SPm4 correction gazing point data. Determine whether there is vertical symmetry data. In other words, the gaze point prediction unit 2341 detects whether or not the correction gazing point SPm1 that is vertically symmetrical with respect to the SPm4 correction gazing point data is detected and the SPm1 correction gazing point data is present (created), or the SPm1 is generated. It is determined whether or not the flag is 1 (step S3133).

  When the SPm1 correction gazing point data exists or the SPm1 creation flag is 1 (step S3133: Yes), the gazing point prediction unit 2341 uses the SPm1 correction gazing point data to perform SPm1 correction. A position that is line-symmetric with respect to the gazing point data for use and the X axis is predicted as the position of the gazing point data for SPm4 correction. That is, the gaze point prediction unit 2341 uses the same X coordinate as the SPm1 correction gaze point data, and uses the Y coordinate as a coordinate line symmetrical to the Y axis of the SPm1 correction gaze point data and the X axis. Viewpoint data is generated (step S3134). Then, the gaze point prediction unit 2341 sets the SPm4 creation flag to 1 (step S3135). Then, the process proceeds to next Step S3136.

  Also, in step S3133, when there is no SPm1 correction gazing point data and the SPm1 creation flag is 0 (step S3133: No), the process proceeds to step S3136.

  In step S3136, the gaze point predicting unit 2341 determines whether or not all correction gaze point creation flags are 1, that is, whether or not correction gaze point data for all landmarks has been obtained (step S3136). S3136). If correction gazing point data for all the marks has not been obtained (step S3136: No), the processing from step S3104 to S3135 is repeatedly executed. Also in the case of newly generating from the generated correction gazing point data as shown in FIG. 29, when all the correction gazing point data are obtained sequentially by the above process (step S3136: Yes), the process ends. .

  As described above, in the present embodiment, when the correction gazing point cannot be detected, the correction characteristic that cannot be detected from the correction gazing point data that is detected using the eye characteristics or the like is used. Since the position of the gazing point is predicted and the predicted position of the correction gazing point is used for correcting the gazing point position, the correction gazing point corresponding to all the landmarks cannot be detected again. A good gazing point position correction can be realized without performing detection.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

100, 2300 Diagnosis support device 101 First display unit 102 Second display unit 103 Speaker 110 Stereo camera 121, 122 Infrared LED light source 130 Drive / IF unit 140 Control unit 141 Display control unit 142 Gaze point detection unit 143, 2343 Area division Unit 144 region specifying unit 145 gazing point correction unit 146 evaluation unit 150 storage unit 2341 gazing point prediction unit

Claims (11)

A gaze point detection device for detecting a position of a gaze point of a detected person,
A display control unit that displays a plurality of landmark correction points at different positions in the target area on the display screen;
A gaze point detection unit for detecting a gaze point of the detected person in the target area;
Based on each position of the correction gazing point that is the gazing point detected corresponding to each mark among the gazing points detected by the gazing point detection unit, the target area is defined as a plurality of first partial areas. An area dividing unit to be divided into
Based on the position of the target gazing point that is the gazing point that is the detection target of the gazing point detection device among the gazing points detected by the gazing point detection unit, the first partial region to which the target gazing point belongs is determined. A first partial region specifying part to be specified;
A second partial region specifying unit that specifies a second partial region that is a region within the target region determined based on the position of the mark corresponding to the correction gazing point that defines the specified first partial region;
Based on the correlation between the position of the first partial region specified by the first partial region specifying unit and the position of the second partial region specified by the second partial region specifying unit, the target gaze point A gazing point detection apparatus comprising a gazing point correction unit that corrects a position.   The gazing point detection apparatus according to claim 1, wherein the display control unit displays four or more marks. In addition to the four or more points, the display control unit further displays a center mark at a predetermined position inside the target region as compared to the four or more points.
The gazing point detection unit further detects the correction gazing point corresponding to the center mark,
The area dividing unit determines a boundary position of the first partial area based on the center mark and the position of the correction gazing point corresponding to the center mark. Gaze point detection device. The first partial region passes through a first point determined based on a correction gazing point corresponding to the center mark and a second point determined based on a correction gazing point corresponding to one of the four points. A region having a first line and a second line passing through the first point as a boundary,
The second partial region is a third straight line passing through the position of the central mark, the position of one of the four points, and a fourth straight line corresponding to the second straight line. A region bounded by the fourth straight line passing through the position,
The gazing point correction unit uses a trigonometric function based on a first angle formed by the first straight line and the second straight line, and a second angle formed by the third straight line and the fourth straight line. The gazing point detection apparatus according to claim 3, wherein the position of the viewpoint is corrected. The gazing point detection unit repeats detection of the gazing point for correction when the gazing point for correction corresponding to the mark is not detected;
The gazing point detection device according to any one of claims 1 to 4. The correction gazing point corresponding to the center mark and the correction gazing point corresponding to at least one of the four or more marks are detected, and a part of the four or more marks is detected. When the correction gazing point corresponding to is not detected, the gazing point that predicts the position of the correction gazing point that has not been detected based on the detected position of the correction gazing point and eye characteristics A prediction unit,
The region dividing unit divides the target region into a plurality of first partial regions based on the position of the correction gazing point and the predicted position of the correction gazing point,
The gazing point detection device according to any one of claims 1 to 4. The gazing point prediction unit has not detected a position that is line-symmetric with respect to a perpendicular line that passes through the position of the detected gazing point for correction and the position of the central gazing point for correction. Predicting the position of
The gazing point detection device according to claim 6. The gazing point prediction unit has not detected a position that is axisymmetric with respect to a horizontal line that passes through the position of the correction gazing point detected and the position of the correction gazing point corresponding to the center mark. Predicting the position of the gazing point,
The gaze point detection apparatus according to claim 6 or 7, A gaze point detection method for detecting a gaze point position of a detected person,
A display control step for displaying a plurality of markers for correcting the gazing point at different positions in the target area on the display screen;
A gazing point detection step of detecting a gazing point of the detected person in the target area;
The target area is defined as a plurality of first partial areas based on the position of the correction gazing point, which is the gazing point detected corresponding to each mark among the gazing points detected in the gazing point detection step. An area dividing step to divide into
Based on the position of the target gazing point that is the gazing point of the detection target of the gazing point detection device among the gazing points detected in the gazing point detection step, the first partial region to which the target gazing point belongs is determined. A first partial region identification step to identify;
A second partial region specifying step of specifying a second partial region that is a region within the target region determined based on the position of the mark corresponding to the correction gazing point that defines the specified first partial region;
Based on the correlation between the position of the first partial area specified in the first partial area specifying step and the position of the second partial area specified in the second partial area specifying step, the target gaze point And a position correction step for correcting the position. A diagnosis support device that supports the diagnosis of developmental disability based on the position of the gaze point of the detected person,
A display control unit that displays a plurality of landmark correction points at different positions in the target area on the display screen;
A gaze point detection unit for detecting a gaze point of the detected person in the target area;
Based on each position of the correction gazing point that is the gazing point detected corresponding to each mark among the gazing points detected by the gazing point detection unit, the target area is defined as a plurality of first partial areas. An area dividing unit to be divided into
Based on the position of the target gazing point that is the gazing point that is the detection target of the gazing point detection device among the gazing points detected by the gazing point detection unit, the first partial region to which the target gazing point belongs is determined. A first partial region specifying part to be specified;
A second partial region specifying unit that specifies a second partial region that is a region within the target region determined based on the position of the mark corresponding to the correction gazing point that defines the specified first partial region;
Based on the correlation between the position of the first partial region specified by the first partial region specifying unit and the position of the second partial region specified by the second partial region specifying unit, the target gaze point A gaze correction unit that corrects the position;
A diagnostic support apparatus comprising: an evaluation unit that calculates an evaluation value indicating a degree of developmental disability based on the corrected position of the target gazing point. A diagnosis support method for supporting a diagnosis of developmental disability based on a gaze position of a detected person,
A display control step for displaying a plurality of markers for correcting the gazing point at different positions in the target area on the display screen;
A gazing point detection step of detecting a gazing point of the detected person in the target area;
The target area is defined as a plurality of first partial areas based on the position of the correction gazing point, which is the gazing point detected corresponding to each mark among the gazing points detected in the gazing point detection step. An area dividing step to divide into
Based on the position of the target gazing point that is the gazing point of the detection target of the gazing point detection device among the gazing points detected in the gazing point detection step, the first partial region to which the target gazing point belongs is determined. A first partial region identification step to identify;
A second partial region specifying step of specifying a second partial region that is a region within the target region determined based on the position of the mark corresponding to the correction gazing point that defines the specified first partial region;
Based on the correlation between the position of the first partial area specified in the first partial area specifying step and the position of the second partial area specified in the second partial area specifying step, the target gaze point A position correction step for correcting the position;
An evaluation step of calculating an evaluation value indicating the degree of developmental disability based on the corrected position of the target gazing point.

Images