JP6217799B2 - Diagnosis support apparatus, diagnosis support method, and diagnosis support program - Google Patents

Description

  The present invention relates to a diagnosis support apparatus, a diagnosis support method, and a diagnosis support program.

Recently, people with developmental disabilities are said to be on the rise. Developmental disorders are known to be detected early and start treatment to reduce symptoms and increase the effect of adapting to society. In Japan, we are aiming for early detection through interviews at the age of 1 and a half. However, there are problems such as shortage of psychiatrists and time-consuming interviews, and the effect is not sufficient. Therefore, there is a need for an objective and efficient diagnosis support device for developmental disorders.

For early detection of developmental disorders, it is ideal to be able to diagnose at the age of 1 and a half.
In addition, it is necessary to consider the use at the time of screening. One characteristic of children with developmental disabilities is that they do not look at the eyes of their opponents. Take a picture of a person's face with the camera,
A method for supporting diagnosis of developmental disorders by applying a method of detecting a gazing point by calculating a corneal reflection and a pupil position has been proposed.

JP 2005-185431 A JP 2008-125619 A JP 2005-198743 A JP 2011-206542 A

In Patent Literature 1, an eye area and a mouth area that reliably includes the mouth of the observation target are specified, the number of frames of a moving image in which the gazing point coordinates are detected in the eye area, and the gazing point coordinates are detected in the mouth area. A method for calculating the number of frames of the moving image, the number of frames of the moving image in which the gaze point coordinates are detected outside the eye area, and the total number of frames to be calculated has been presented. A method was sought.

  The present invention has been made in view of the above, and an object thereof is to provide a diagnosis support apparatus, a diagnosis support method, and a diagnosis support program that can improve the accuracy of diagnosis.

In order to solve the above-described problems and achieve the object, the present invention provides a display unit, an imaging unit that images a subject, and a line of sight that detects the gaze direction of the subject from a captured image captured by the imaging unit. A detection unit; a viewpoint detection unit that detects the viewpoint of the subject in the display region of the display unit based on the line-of-sight direction; a first image used for calibration of parameters for detection of the line-of-sight direction; and the viewpoint The second used for correcting the position of the viewpoint detected by the detection unit
And an output control unit that causes the display unit to display an image and a third image used for diagnosis in this order.

  The diagnosis support apparatus, diagnosis support method, and diagnosis support program according to the present invention have an effect of improving the accuracy of diagnosis.

FIG. 1 is a diagram illustrating an example of an arrangement of a display unit, a stereo camera, and a light source used in the first embodiment. FIG. 2 is a diagram illustrating an outline of functions of the diagnosis support apparatus according to the first embodiment. FIG. 3 is a block diagram illustrating an example of detailed functions of the respective units illustrated in FIG. FIG. 4 is a diagram illustrating an example of eye and distance detection when two cameras are used. FIG. 5 is a diagram illustrating an example of a target point mark. FIG. 6 is a diagram illustrating SP0a to SP4a. FIG. 7 is a diagram showing the frame F and the frame Fe. FIG. 8 is a diagram showing a partial region of the frame Fe. FIG. 9 is a diagram illustrating a partial region of the frame F. FIG. 10 is a diagram showing the coordinates of each point. FIG. 11 is a diagram showing the lower quadrant A of the first quadrant. FIG. 12 is a diagram illustrating the upper quadrant H of the fourth quadrant. FIG. 13 is a diagram showing the upper quadrant area C of the second quadrant. FIG. 14 is a diagram showing the first quadrant upper area B. As shown in FIG. FIG. 15 is a flowchart illustrating an example of a diagnosis support process according to the first embodiment. FIG. 16 is a diagram illustrating an example of a menu screen. FIG. 17 is a diagram illustrating an example of the analysis menu screen. FIG. 18 is a diagram illustrating an example of the position adjustment image. FIG. 19 is a diagram illustrating another example of the position adjustment image. FIG. 20 is a diagram illustrating another example of the position adjustment image. FIG. 21 is a diagram illustrating an example of an image for calibration. FIG. 22 is a diagram illustrating an example of an image for calibration. FIG. 23 is a flowchart showing details of the correction gazing point detection processing. FIG. 24 is a diagram for explaining a data distribution calculation process. FIG. 25A is a diagram for explaining the target point mark display process in the correction gazing point detection process. FIG. 25-2 is a diagram for explaining target point mark display processing in correction gazing point detection processing. FIG. 25C is a diagram for explaining target point mark display processing in the correction gazing point detection processing. FIG. 25D is a diagram for explaining the target point mark display process in the correction gazing point detection process. FIG. 25-5 is a diagram for explaining the target point mark display processing in the correction gazing point detection processing. FIG. 26 is a diagram illustrating an example of a target point mark. FIG. 27 is a diagram illustrating an example of a target point mark. FIG. 28 is a diagram illustrating an example of a target point mark. FIG. 29 is a diagram illustrating an example of a target point mark. FIG. 30 is a diagram illustrating an example of a target point mark. FIG. 31 is a diagram illustrating an example of a target point mark. FIG. 32 is a diagram illustrating an example of a target point mark. FIG. 33 is a diagram illustrating an example of a target point mark. FIG. 34 is a diagram illustrating an example of a target point mark. FIG. 35 is a diagram illustrating an example of a target point mark. FIG. 36 is a diagram illustrating an example of a target point mark. FIG. 37 is a flowchart illustrating an example of analysis processing according to the first embodiment. FIG. 38 is a flowchart showing a process of specifying a partial area in the frame Fe in the partial area specifying process. FIG. 39 is a diagram illustrating an example of a diagnostic image. FIG. 40 is a diagram illustrating an example of a diagnostic image. FIG. 41 is a diagram illustrating an example of a diagnostic image. FIG. 42 is a diagram illustrating an example of a diagnostic image. FIG. 43 is a diagram illustrating an example of a diagnostic image. FIG. 44 is a diagram illustrating an example of a diagnostic image. FIG. 45A is a diagram illustrating an example of a diagnostic image. FIG. 45-2 is a diagram illustrating an example of a diagnostic image. FIG. 46 is a diagram illustrating an example of an eye catch image. FIG. 47 is a diagram illustrating an example of an eye catch image. FIG. 48 is a diagram illustrating an example of an eye catch image. FIG. 49 is a diagram showing the timing of displaying an eye catch image and outputting sound. FIG. 50 is a diagram illustrating an example of a result display screen. FIG. 51 is a diagram illustrating an example of a result display screen. FIG. 52 is a diagram illustrating an example of a result display screen. FIG. 53 is a diagram illustrating an example of a result display screen. FIG. 54 is a diagram illustrating an example of a result display screen. FIG. 55 is a diagram illustrating an example of a display method of a gazing point. FIG. 56 is a diagram illustrating an example of a display method of a gazing point. FIG. 57 is a diagram illustrating an example of a display method of a gazing point. FIG. 58 is a diagram illustrating an example of a display method of a gazing point.

Hereinafter, embodiments of a diagnosis support apparatus and a diagnosis support method according to the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

(First embodiment)
FIG. 1 is a diagram illustrating an example of an arrangement of a display unit, a stereo camera, and a light source used in the first embodiment. As shown in FIG. 1, in the first embodiment, a set of stereo cameras 102 is arranged below the display screen 101. The stereo camera 102 is an imaging unit that can perform stereo shooting with infrared rays, and includes a right camera 202 and a left camera 204.

Immediately before each lens of the right camera 202 and the left camera 204, infrared LEDs (
Light Emitting Diode) light sources 203 and 205 are arranged respectively. The infrared LED light sources 203 and 205 include an inner peripheral LED and an outer peripheral LED having different wavelengths for emitting light. The pupils of the subject are detected by the infrared LED light sources 203 and 205. As a pupil detection method, for example, the method described in Patent Document 2 can be applied.

When detecting the line of sight, the space is expressed by coordinates to identify the position. In this embodiment, the center position of the display screen 101 is the origin, the top and bottom are the Y coordinate (up is +), the side is the X coordinate (right is +), and the depth is the Z coordinate (front is +). .

FIG. 2 is a diagram illustrating an outline of functions of the diagnosis support apparatus 100. FIG. 2 shows a part of the configuration shown in FIG. 1 and a configuration used for driving the configuration. As shown in FIG. 2, the diagnosis support apparatus 100 includes a right camera 202, a left camera 204, infrared LED light sources 203 and 205, a speaker 105, a drive / IF unit 208, a control unit 300, and a storage unit. 150
And a display unit 210. In FIG. 2, the display screen 101 shows the positional relationship between the right camera 202 and the left camera 204 in an easy-to-understand manner.
It is a screen displayed at 0.

The speaker 105 functions as an audio output unit that outputs an audio or the like for alerting the subject during calibration.

The drive / IF unit 208 drives each unit included in the stereo camera 102. The drive / IF unit 208 serves as an interface between each unit included in the stereo camera 102 and the control unit 300.

The storage unit 150 stores various information such as a control program, a measurement result, and a diagnosis support result.
The storage unit 150 stores, for example, an image to be displayed on the display unit 210. The display unit 210
Various information such as a target image for diagnosis is displayed.

FIG. 3 is a block diagram illustrating an example of detailed functions of the respective units illustrated in FIG. As shown in FIG. 3, a display unit 210 and a drive / IF unit 208 are connected to the control unit 300. Drive / IF
The unit 208 includes camera IFs 314 and 315, an LED drive control unit 316, a speaker drive unit 322, and a printer IF 323.

The right camera 2 is connected to the drive / IF unit 208 via the camera IFs 314 and 315, respectively.
02, the left camera 204 is connected. The driving / IF unit 208 drives these cameras to image the subject.

A frame synchronization signal is output from the right camera 202. The frame synchronization signal is input to the left camera 204 and the LED drive control unit 316. Thus, in the first frame, the left and right wavelength 1 infrared light sources (wavelength 1-LED 303, wavelength 1-LED 30 are shifted in timing).
5) is emitted, and images corresponding to the left and right cameras (the right camera 202 and the left camera 204) are captured correspondingly, and the left and right wavelength 2 infrared light sources (wavelength 2-LED304, wavelength 2) are shifted in timing in the second frame. -The LED 306) is caused to emit light, and correspondingly, images from the left and right cameras are captured.

The infrared LED light source 203 includes a wavelength 1-LED 303 and a wavelength 2-LED 304. Infrared LED light source 205 includes wavelength 1-LED 305, wavelength 2-LED 306,
It has.

Wavelength 1-LEDs 303 and 305 emit infrared light having wavelength 1. Wavelength 2-LED304
, 306 irradiate infrared rays having a wavelength of 2.

The wavelength 1 and the wavelength 2 are, for example, a wavelength of less than 900 nm and a wavelength of 900 nm or more, respectively. When the reflected light reflected by the pupil is irradiated with infrared light having a wavelength of less than 900 nm, a bright pupil image is obtained as compared with the case where reflected light reflected by the pupil is irradiated with infrared light having a wavelength of 900 nm or more. It is because it is obtained.

The speaker driving unit 322 drives the speaker 105. The printer IF 323 is an interface for connecting to a printer 324 as a printing unit. The printer 324
May be provided inside the diagnosis support apparatus 100.

The control unit 300 controls the entire diagnosis support apparatus 100. The control unit 300 includes a line-of-sight detection unit 3
51, viewpoint detection unit 352, distance detection unit 353, output control unit 354, calibration unit 355
A region dividing unit 356, a region specifying unit 357, a correcting unit 358, and an evaluating unit 359.

The gaze detection unit 351 detects the gaze (gaze direction) of the subject from the captured image captured by the imaging unit (stereo camera 102). The process of detecting the line of sight includes a process of detecting the eye position of the subject (position detection unit). The viewpoint detection unit 352 detects the viewpoint of the subject using the detected gaze direction. For example, the viewpoint detection unit 352 detects a viewpoint (gaze point) that is a point that the subject gazes out of the target image displayed on the display screen 101. Line-of-sight detection unit 3
As a gaze detection method by 51 and a viewpoint detection method by the viewpoint detection unit 352, any conventionally used method can be applied. Below, similarly to patent document 3, it demonstrates as an example the case where a test subject's gaze direction and a gaze point are detected using a stereo camera.

In this case, the gaze detection unit 351 first detects the gaze direction of the subject from the image captured by the stereo camera 102. The gaze detection unit 351 detects the gaze direction of the subject using, for example, the methods described in Patent Documents 1 and 2. Specifically, the line-of-sight detection unit 351 obtains the difference between the image captured by irradiating the infrared light having the wavelength 1 and the image captured by irradiating the infrared light having the wavelength 2, and the image in which the pupil image is clarified. Generate. The line-of-sight detection unit 351 uses the two images generated as described above from the images captured by the left and right cameras (the right camera 202 and the left camera 204) and uses the stereo vision technique to determine the position of the subject's pupil ( Eye position) is calculated. The line-of-sight detection unit 351 calculates the position of the subject's corneal reflection using images taken by the left and right cameras. Then, the gaze detection unit 351 calculates a gaze vector representing the gaze direction of the subject from the position of the pupil of the subject and the corneal reflection position.

In addition, the detection method of the eye position and the line of sight of the subject is not limited to this. For example, the eye position and line of sight of the subject may be detected by analyzing an image captured using visible light instead of infrared light.

The viewpoint detection unit 352 detects, for example, the intersection of the line-of-sight vector represented in the coordinate system as shown in FIG. 1 and the XY plane as the gaze point of the subject. When the gaze direction of both eyes is obtained, the gaze point may be measured by obtaining the intersection of the left and right gazes of the subject.

The distance detection unit 353 detects the distance from the imaging unit (stereo camera 102) to the eye position of the subject. In the present embodiment, the distance detection unit 353 determines the distance dz in the depth direction (Z coordinate direction) between the stereo camera 102 and the subject's eyes, and the distance from the imaging unit (stereo camera 102) to the position of the subject's eyes. Detect as.

FIG. 4 is a diagram illustrating an example of eye and distance detection when two cameras (the right camera 202 and the left camera 204) are used. For the two cameras, a camera calibration theory based on a stereo calibration method is applied in advance to obtain camera parameters. The stereo calibration method is
Any conventional method such as a method using Tsai's camera calibration theory can be applied. Using the eye position detected from the image captured by the right camera 202, the eye position detected from the image captured by the left camera 204, and the camera parameters, the three-dimensional coordinates of the eye in the world coordinate system are obtained. It is done. Thereby, the distance between the eyes and the stereo camera 102 and the pupil coordinates can be estimated. The pupil coordinate is a coordinate value representing the position of the subject's eye (pupil) on the XY plane. The pupil coordinates can be, for example, coordinate values obtained by projecting the eye position expressed in the world coordinate system onto the XY plane. Usually, the pupil coordinates of the left and right eyes are obtained.

A marker 402 and an eye position image 403 are displayed on the display screen 101. The eye position image 403 is an image showing the eye position of the subject. The marker 402 is an image corresponding to the size of the eye position image 403 of the subject at a predetermined reference distance. Although a rectangular marker is displayed in FIG. 4, the shape of the marker 402 is not limited to a rectangle.
For example, a polygonal, circular, or elliptical marker other than a rectangle (rectangle) may be used.

When a coordinate system (world coordinate system) as shown in FIG. 1 is used, the center position of the display screen 101 is the origin. For this reason, the Z coordinate value of the detected eye position corresponds to the distance dz in the depth direction between the stereo camera 102 and the eye of the subject. The actual stereo camera 102
And the distance between the eyes calculated by the eye gaze detection unit 351 may be calculated. For example, the viewpoint detection unit 352 includes the right camera 202 and the left camera 204.
The distance to the eyes of the subject may be detected from any one of the above positions or an intermediate position between the position of the right camera 202 and the position of the left camera 204.

Returning to FIG. 3, the output control unit 354 controls output of various types of information to the display unit 210, the speaker 105, the printer 324, and the like. For example, the output control unit 354 includes an image (first image) used for calibration of parameters for detecting the line-of-sight direction (first image) and an image used for detection of a correction gazing point for correcting the position of the gazing point ( Second image), an image used for diagnosis (diagnostic image (third image)), and display unit 2 such as an evaluation result by evaluation unit 359
Control the output for 10.

As will be described later, the output control unit 354 may control the display unit 210 to display an image used for calibration, an image used for detecting a correction gazing point, and a diagnostic image in this order. Thereby, each process required for a diagnosis can be performed smoothly and the precision of a diagnosis can be improved.

The output control unit 354 causes the display unit 210 to display an image for adjusting the position of the subject (position adjustment image). For example, the output control unit 354 uses at least one of the position image of the subject's eyes, the reference image indicating the range of the reference region, and the imaging range image indicating the range of the imaging region as a position adjustment image, The display mode is changed on the display screen 101 according to the positional relationship with the eye position of the subject. For example, the output control unit 354 changes the size of the eye position image 403 according to the distance and causes the display screen 101 to display the size. The display mode is not limited to the size. For example, color, image tone, image brightness, and
It may be a character, symbol or figure included in the image.

The reference area is an area included in the imaging area, and is determined in advance as an area representing an appropriate position range of the eyes of the subject, for example. For example, an area having a predetermined size within the imaging area including the center of the imaging area can be set as the reference area. The setting area represents an appropriate area for detecting the position of the eye of the subject. For example, the reference area may be the setting area, and the imaging area may be the setting area.

The reference image is an image that is displayed on the display screen 101 as an image representing a range corresponding to the reference region. The reference image is displayed at the center of the display screen 101, for example. For example, the output control unit 354 changes the color of the reference image according to the positional relationship and causes the display screen 101 to display the color. Note that the display mode is not limited to colors, and brightness and the like may be changed.

The output control unit 354 may display a moving image in the reference image. As the moving image, for example, an image in which a character imitating a person or an animal moves can be used. Thereby, a test subject can be made to pay attention to the display screen 101, and a test subject's position can be adjusted appropriately.

Further, the output control unit 354 causes the display screen 101 to display a mark of the target gazing point position (target point) used for the gazing point position correction process. Here, the target point is a position to be watched by the subject.

FIG. 5 is a diagram illustrating an example of a target point mark. As shown in FIG. 5, a circle image having a predetermined radius may be used as a mark. Further, as a mark, a frame having a predetermined shape and an image (still image or moving image) displayed in the frame may be used. In the present embodiment, the output control unit 35
4 displays five marks (SP0 to SP4) indicating five different target points in the display area on the display unit 210. The shape and number of landmarks are not limited to these.

In the present embodiment, the vertex positions of a quadrilateral having four sides parallel to each side of a rectangular screen (display area) are set as target points SP1 to SP4. In the center of the screen,
The target point of SP0. That is, SP0 is SP1 to SP1 in the display area.
It is a center mark which shows a position inside compared with SP4. In the present embodiment, SP0
Is the same as the origin of the XYZ coordinate system, that is, the center position of the screen of the display unit 210. It is assumed that the type and display position of the target point mark image are set in advance in the storage unit 150 or the like.

The mark of the target point may be an image that is different in brightness and saturation from other regions of the display unit 210, and may be an image other than a circular image. As another example, the target point mark may be indicated by irradiating light to the target point arrangement position. That is, the mark may be light other than the image. Thus, the mark may be in any form that allows the subject to recognize the position to be watched, and the display form is not limited to the embodiment.

Returning to FIG. 3, the calibration unit 355 calibrates the calculation parameters used for detecting the gaze direction (calibration for gaze detection). Generally, the optical axis of the eye and the line-of-sight direction are different for each individual. For this reason,
This is because it is necessary to correct the optical axis and the line-of-sight direction to coincide. In the gaze detection calibration process, the subject is gazes at a predetermined position (for example, an image used for gaze detection calibration) known on a predetermined coordinate system, and the pupil center (pupil position) and corneal reflection position at that time And measure. Then, the calibration unit 355 corrects the calculation parameter for eye gaze detection so that the eye gaze direction obtained from the measured pupil position and corneal reflection position is directed to a predetermined position.

The area dividing unit 356 divides the screen of the display unit 210 into a plurality of partial areas based on the gazing point (correction gazing point) detected with respect to the target point mark displayed for correction.

The area specifying unit 357 specifies which partial area in the display area the gazing point (analysis gazing point) detected for the diagnostic image belongs to.

The correcting unit 358 corrects the position of the analysis gazing point based on the partial region to which the analysis gazing point belongs.

Here, the processing of the region dividing unit 356, the region specifying unit 357, and the correcting unit 358 is described with reference to FIG.
Details will be described with reference to FIG.

Points SP0a to SP4a shown in FIG. 6 are target point marks SP0 to SP, respectively.
4 is a correction gaze point of the subject obtained by gaze 4.

The area dividing unit 356 first matches the point SP0a with the origin of the XY coordinate system. In particular,
The area dividing unit 356 calculates an origin displacement value from the origin of the XY coordinate system to SP0a. Here, the origin and the position of SP0 coincide. Then, the area dividing unit 356 includes SP1a to SP4a.
The coordinates of SP1b to SP4b, each of which is moved by the origin displacement value, are obtained. For example, S
When the coordinates of P0a are (x0, y0), the origin displacement value is (−x0, −y0). Therefore, the area dividing unit 356 adds (−x0, SP4a) to the coordinates of SP1a to SP4a.
The coordinates of SP1b to SP4b are obtained by adding -y0).

FIG. 7 is a diagram showing the frame F and the frame Fe after the point SP0a in FIG. 6 is moved to the origin of the XY coordinate system. The frame F is a frame (frame) determined by the target point points F1 to F4. Points F1 to F4 correspond to points SP1 to SP4, respectively. The frame Fe is points Fe1 to F corresponding to the correction gazing points detected corresponding to the target points.
This is a frame determined by e4. Points Fe1 to Fe4 are points SP1b to SP1, respectively.
It corresponds to P4b. That is, the frame F is a reference frame corresponding to the XY coordinate system of the display screen, and the frame Fe is an error frame including a subject-specific error and the like. Note that the frame F and the frame Fe have different concepts from the frames to be imaged by the left and right cameras (the right camera 202 and the left camera 204).

The gaze point for analysis calculated by the viewpoint detection unit 352 is a point on the frame Fe. Furthermore, due to the aspherical shape of the eyeball of the subject, the degree of error varies depending on the line-of-sight direction, that is, the mark position. Therefore, in the diagnosis support apparatus 100 according to the present embodiment, the target area is divided into eight partial areas, and the frame Fe of the gazing point for analysis is divided into partial areas.
By converting the upper coordinate value to the coordinate value on the frame F, the gaze point for analysis is corrected.

The area dividing unit 356 divides the target area into eight partial areas based on the positions of the points Fe1 to Fe4. FIG. 8 is a diagram showing eight partial areas. Each partial area has an origin O and a point Fe1.
This is a region determined by Fe4, X axis, and Y axis. Here, the points Fe1 to Fe4 are points corresponding to SP1b to SP4b, respectively. SP1b to SP4b are respectively SPs.
This is a point determined based on 1a to SP4a. In other words, the region dividing unit 356 is configured so that SP1a ~
Based on the position of SP4a, the target area is divided into eight partial areas.

Here, the intersection of the straight line passing through the points Fe1 and Fe4 and the X axis is defined as a point Fe5. Point Fe1
And the point of intersection of the straight line passing through the point Fe2 and the Y axis is a point Fe6. A point of intersection between the straight line passing through the points Fe2 and Fe3 and the X axis is defined as a point Fe7. A point of intersection between the straight line passing through the points Fe3 and Fe4 and the Y axis is defined as a point Fe8.

The area dividing unit 356 includes a first quadrant lower area A, a first quadrant upper area B, a second quadrant upper area C, a second quadrant lower area D, a third quadrant upper area E, and a third quadrant shown in FIG. The area is divided into eight partial areas of a lower area F, a fourth quadrant lower area G, and a fourth quadrant upper area H. Further, the area specifying unit 3
57, the analysis gazing point detected by the viewpoint detection unit 352 is based on the coordinates of the analysis gazing point and the positions of the eight partial regions divided by the region dividing unit 356. It is specified to which partial region the point belongs.

Here, the lower quadrant A of the first quadrant includes a straight line passing through the origin O and the point Fe1, and the origin O and the point Fe5.
Is a region having a straight line (X axis) passing through the boundary position. Upper quadrant B of first quadrant has origin O and point F
This is an area having a straight line passing through e1 and a straight line passing through the origin O and the point Fe6 (Y axis) as boundary positions.

The second quadrant upper area C is an area having a straight line passing through the origin O and the point Fe2 and a straight line passing through the origin O and the point Fe6 (Y axis) as boundary positions. The lower quadrant area D is an area having a boundary position that is a straight line passing through the origin O and the point Fe2 and a straight line passing through the origin O and the point Fe7 (X axis).

The third quadrant upper area E is an area having a straight line passing through the origin O and the point Fe3 and a straight line passing through the origin O and the point Fe7 (X axis) as boundary positions. The lower quadrant F of the third quadrant is an area having a boundary position that is a straight line passing through the origin O and the point Fe3 and a straight line passing through the origin O and the point Fe8 (Y axis).

The lower quadrant region G is a region having a boundary position that is a straight line passing through the origin O and the point Fe4 and a straight line passing through the origin O and the point Fe8 (Y axis). The upper quadrant H of the fourth quadrant is an area having a straight line passing through the origin O and the point Fe4 and a straight line passing through the origin O and the point Fe5 (X axis) as boundary positions.

FIG. 9 shows a first quadrant lower area A, a first quadrant upper area B, a second quadrant upper area C, a second quadrant lower area D, a third quadrant upper area E, a third quadrant lower area F, It is a figure which shows the partial area | region (A'-H ') of the flame | frame F corresponding to 4th quadrant lower area | region G and 4th quadrant upper area | region H, respectively.

First quadrant lower area A ′, first quadrant upper area B ′, second quadrant upper area C ′, second quadrant lower area D ′, third quadrant upper area E ′, third quadrant lower area F ′ , The fourth quadrant lower region G ′ and the fourth quadrant upper region H ′ are respectively a first quadrant lower region A, a first quadrant upper region B, a second quadrant upper region C, a second quadrant lower region D, This is a region corresponding to the third quadrant upper region E, the third quadrant lower region F, the fourth quadrant lower region G, and the fourth quadrant upper region H.

The area specifying unit 357 specifies the partial area on the frame F corresponding to the specified partial area on the frame Fe, in addition to specifying the partial area on the frame Fe to which the gazing point for analysis belongs as described above. To do. For example, when the first quadrant lower area A on the frame Fe is specified,
The first quadrant lower area A ′ on the frame F is specified as the corresponding partial area. That is,
The region specifying unit 357 specifies a partial region on the frame Fe and a corresponding partial region on the frame F, and corresponds to a first partial region specifying unit and a second partial region specifying unit. .

The partial area on the frame F will be described with reference to FIG. A point of intersection between the straight line passing through the points F1 and F4 and the X axis is defined as a point F5. A point of intersection between the straight line passing through the points F1 and F2 and the Y axis is defined as a point F6. A point of intersection between the straight line passing through the points F2 and F3 and the X axis is defined as a point F7. Point F3 and point F
Let the intersection of the straight line passing through 4 and the Y axis be point F8.

The first quadrant lower area A ′ is an area having a boundary position that is a straight line passing through the origin O and the point F1 and a straight line passing through the origin O and the point F5 (X axis). The first quadrant upper area B ′ is an area having a boundary position that is a straight line passing through the origin O and the point F1 and a straight line passing through the origin O and the point F6 (Y axis).

The second quadrant upper area C ′ is an area having a boundary position that is a straight line passing through the origin O and the point F2 and a straight line passing through the origin O and the point F6 (Y axis). The second quadrant lower region D ′ is a region having a boundary position that is a straight line passing through the origin O and the point F2 and a straight line passing through the origin O and the point F7 (X axis).

The third quadrant upper area E ′ is an area having a boundary position that is a straight line passing through the origin O and the point F3 and a straight line passing through the origin O and the point F7 (X axis). The lower quadrant F ′ of the third quadrant is an area having a boundary position that is a straight line passing through the origin O and the point F3 and a straight line passing through the origin O and the point F8 (Y axis).

The fourth quadrant lower region G ′ is a region having a boundary position that is a straight line passing through the origin O and the point F4 and a straight line passing through the origin O and the point F8 (Y axis). The fourth quadrant upper area H ′ is an area having a boundary position that is a straight line passing through the origin O and the point F4 and a straight line passing through the origin O and the point F5 (X axis).

The correcting unit 358 corrects the position of the gazing point based on the correlation between the partial area on the frame Fe and the corresponding partial area in the frame F, that is, the displacement value from the frame Fe to the frame F.

Hereinafter, the processing of the correction unit 358 will be described in detail with reference to FIGS. FIG. 10 is a diagram showing the coordinates of each point. As shown in FIG. 10, the coordinates of the point F1 are (x4, y4), the coordinates of the point F2 are (x10, y4), the coordinates of the point F3 are (x10, y8), and the coordinates of the point F4 are (x
4, y8), the coordinates of the point F5 are (x4, 0), the coordinates of the point F6 are (0, y4), the coordinates of the point F7 are (x10, 0), and the coordinates of the point F8 are (0, y8). . Also, the coordinates of the point Fe1 are (x6
, Y6), the coordinates of the point Fe2 are (x11, y11), and the coordinates of the point Fe3 are (x13, y13).
The coordinates of the point Fe4 are (x9, y9), the coordinates of the point Fe5 are (x7,0), the coordinates of the point Fe6 are (0, y7), the coordinates of the point Fe7 are (x20,0), and the coordinates of the point Fe8 are (0, y20).

Further, as shown in FIG. 11, the point of interest for analysis detected in the first quadrant lower area A is a point P2A, and the coordinates of the point P2A are (x2A, y2A). Further, the point of interest for analysis after the correction of the point P2A is the point P0A, and the coordinates of the point P0A are (x0A, y0A).

Further, a straight line passing through the points Fe1 and Fe5 is L1A, and a straight line passing through the points F1 and F5 is L2A.
And A straight line passing through the point P2A and the origin O is L3A, a straight line passing through the point P0A and the origin is L4A, a straight line passing through the point Fe1 and the origin is L5A, and a straight line passing through the point F1 and the origin is L6A.

Further, the intersection of the straight line L2A and the straight line L4A is defined as a point P1A, and the coordinates of the point P1A are represented by (x4, y1
A). Further, an intersection of the straight line L1A and the straight line L3A is a point P3A, and the coordinates of the point P3A are (
x3A, y3A).

Further, the distance between the origin and the point P0A is d0A, the distance between the origin and the point P1A is d1A, the distance between the origin and the point P2A is d2A, and the distance between the origin and the point P3A is d3A.

The angle 0A formed by the straight line L3A and the X axis is θ0A, and the angle 1A formed by the straight line L5A and the X axis is 1A.
Is θ1A, the angle 2A formed by the straight line L4A and the X axis is θ2A, and the angle 3A formed by the straight line L6A and the X axis is θ3A.

Under the above conditions, the correction unit 358 assumes that the ratio of the angle θ0A of the angle 0A to the angle θ1A of the angle 1A is equal to the ratio of the angle 2A of the angle 2A to the angle θ3A of the angle 3A.
Further, the corrected position of the gazing point for correction is calculated by determining the distance d0A on the assumption that the ratio of the distance d2A to the distance d3A is equal to the ratio of the distance d0A to the distance d1A.

First, the correction unit 358 calculates the angle θ0A, the angle θ1A, and the angle θ3A using (Expression 1) to (Expression 3).

The ratio of the angle 0A to the angle θ1A of the angle 1A and the angle θ1A of the angle 2A and the angle 3 of the angle θ2A of the angle 2A
Since the ratio of A to the angle θ3A is equal, (Equation 4) holds. The correction unit 358
The angle θ0A, the angle θ1A, and the angle θ3A calculated by (Expression 1) to (Expression 3) in (Expression 4).
Is substituted for angle θ2A.

The correcting unit 358 also assumes that the point P3A is an intersection of the straight line L1A and the straight line L3A, and the point P3
The coordinates (x3A, y3A) of A are calculated. Here, the straight line L1A is represented by (Equation 5).
(Expression 5) can be expressed as (Expression 6).
In (Formula 6), al and bl are represented by (Formula 7) and (Formula 8), respectively.
The straight line L3A is represented by (Equation 9).
By substituting (Equation 9) into (Equation 6), (Equation 10) and (Equation 11) are obtained.
The correction unit 358 calculates the coordinates (x3A, y3A) of the point P3A according to (Expression 10) and (Expression 11).
) Is calculated.

Further, the correction unit 358 calculates a ratio k0 of the distance d2A to the distance d3A by (Equation 12).

The correcting unit 358 also calculates the coordinates (x4, y1A) of the point P1A. Here, straight line L2
A is represented by (Equation 13).
Using (Expression 13), (Expression 14) and (Expression 15) are obtained.
Based on the above, the correction unit 358 determines the coordinates of the point P1A (Equation 14) and (Equation 15) (
x4, y1A) is calculated.

Further, the correction unit 358 calculates the distance d1A by (Equation 16).

The correction unit 358 assumes that the point P0A is a point where the ratio of the distance d0A to the distance d1A is k0, and the coordinates (x0A, y0A) of the point P0A are expressed by (Expression 17) and (Expression 18).
Is calculated.

Note that the correction unit 358 also corrects the position of the analysis gazing point belonging to each partial region in the other partial regions using the same arithmetic expression.

FIG. 12 is a diagram illustrating the upper quadrant H of the fourth quadrant. When the analysis gazing point belongs to the upper quadrant H of the fourth quadrant, the angles 0B to 3B are defined with reference to the X axis, and straight lines L1B to L6B are defined. Then, the correction unit 358 determines the angle θ2B of the angle 2B, assuming that the ratio of the angle 0B to the angle θ1B of the angle 1B and the angle θ2B of the angle 2B to the angle θ3B of the angle 3B is equal. Furthermore, the correction unit 358 calculates the distance d0B on the assumption that the ratio of the distance d2B to the distance d3B is equal to the ratio of the distance d0B to the distance d1B. Thereby, the correcting unit 358 corrects the analysis gazing point P2B (x2B, y2B) after the analysis gazing point P2B (x2B, y2B).
The coordinates (x0B, y0B) of 0B are calculated.

FIG. 13 is a diagram showing the upper quadrant area C of the second quadrant. When the analysis gazing point belongs to the second quadrant upper area C, the angles 0C to 3C are defined with reference to the Y axis, and the straight lines L1C to L6C are defined. Then, the correction unit 358 obtains the angle θ2C of the angle 2C, assuming that the ratio of the angle 0C to the angle θ1C of the angle 1C is equal to the ratio of the angle 2C of the angle 2C to the angle θ3C of the angle 3C. Further, the correction unit 358 calculates the distance d0C on the assumption that the ratio of the distance d2C to the distance d3C is equal to the ratio of the distance d0C to the distance d1C. As a result, the correcting unit 358 corrects the analysis attention point P2C (x2C, y2C) after the analysis attention point P2C (x2C, y2C).
The coordinates of 0C (x0C, y0C) are calculated.

FIG. 14 is a diagram showing the first quadrant upper area B. As shown in FIG. When the analysis gazing point belongs to the first quadrant upper area B, the angles 0D to 3D are determined with reference to the Y axis, and the straight lines L1D to L6D are determined. Then, the correction unit 358 determines the angle θ2D of the angle 2D, assuming that the ratio of the angle 0D to the angle θ1D of the angle 1D is equal to the ratio of the angle 2D of the angle 2D to the angle θ3D of the angle 3D. Further, the correction unit 358 calculates the distance d0D on the assumption that the ratio of the distance d2D to the distance d3D is equal to the ratio of the distance d0D to the distance d1D. Thereby, the correcting unit 358 corrects the analysis gazing point P2D (x2D, y2D) after the analysis gazing point P2D (x2D, y2D).
The coordinates (x0D, y0D) of 0D are calculated.

Returning to FIG. 3, the evaluation unit 359 calculates an evaluation value as an index related to the degree of developmental disability based on the diagnostic image and the corrected attention point for analysis. The evaluation unit 359, for example, based on the position of the subject's point of interest for analysis when displaying a diagnostic image such as FIGS.
Calculate the rate of looking at the eyes of the face image as a diagnostic image as the evaluation value, the lower the evaluation value,
Calculate an evaluation value that indicates a high probability of developmental disability. The evaluation unit 359 may calculate the evaluation value based on the diagnostic image and the analysis point of interest, and the calculation method is not limited to the embodiment.

Next, diagnosis support processing by the diagnosis support apparatus 100 according to the first embodiment configured as described above will be described with reference to FIG. FIG. 15 is a flowchart illustrating an example of a diagnosis support process according to the first embodiment.

First, the output control unit 354 displays a menu screen (step S1). FIG. 16 is a diagram illustrating an example of a menu screen. As shown in FIG. 16, the menu screen has an analysis button 5
01, a result display button 502, and a program end button 503. When the analysis button 501 is pressed, the output control unit 354 causes the display unit 210 to display an analysis menu screen that is a screen for executing an analysis process using a diagnostic image. When the result display button 502 is pressed, the output control unit 354 causes the display unit 210 to display a result display screen (described later).
When the program end button 503 is pressed, the output control unit 354 ends the diagnosis support process.

FIG. 17 is a diagram illustrating an example of the analysis menu screen. As shown in FIG. 17, the analysis menu screen includes a “start analysis” button 601, a “return to basic menu” button 602,
A pull-down 611 for selecting calibration content, a pull-down 612 for selecting analysis content, and an information input field 613 are included.

When the “start analysis” button 601 is pressed, processing using a diagnostic image (processing after step S2 in FIG. 15) is started. When the “return to basic menu” button 602 is pressed, the menu screen of FIG. 16 is displayed again. The calibration content is the calibration unit 355.
It represents contents such as images used in calibration (calibration for eye-gaze detection).
An example of the calibration content is shown in FIGS. The content selected using the pull-down 611 is used for calibration by the calibration unit 355. The analysis content represents content such as an image (diagnosis image) used for diagnosis. The content selected using the pull-down 612 is used as a diagnostic image. The information input column 613 is a column for inputting various types of information necessary for diagnosis such as subject information.

Returning to FIG. When the start of analysis is instructed, for example, the analysis button 50 is displayed on the menu screen.
When 1 is pressed and the “Start Analysis” button 601 is pressed on the analysis menu screen, the output control unit 354 causes the display unit 210 to display a position adjustment image (step S2).

FIG. 18 is a diagram illustrating an example of the position adjustment image 701. The position adjustment image 701 includes a scale 703 as a reference image, a frame 702 as an imaging range image, and an eye position image 70.
5, a marker 704, and a character image 706 that is a moving image. In the example of FIG. 18, since the subject is located at an appropriate distance, the diameter of the eye position image 705 and the width of the marker 704 are displayed to match. In this case, the output control unit 354 may display, for example, the eye position image 705 in a color (for example, green) indicating that the distance is appropriate. Further, FIG.
In the example, since the subject exists in the reference area, the output control unit 354 may display the scale 703 in a color (for example, green) indicating that it is in the reference area. When it is detected that at least one of the position images 705 of the left and right eyes of the subject is outside the reference region, the output control unit 3
54 may display the scale 703 in a color (for example, red) indicating that it is outside the reference region. In addition, when the output control unit 354 detects that at least one of the position images 705 of the left and right eyes of the subject is outside the reference region, the output control unit 354 outputs a sound such as “more right” or “more left”. May be. The output control unit 354 displays the character image 706 so that the position of the eyes (the direction of the line of sight) and the shape of the character image 706 change with time in the scale 703. The moving image is not the character image 706 but may be anything that attracts attention.
For example, as a moving image, an image like fireworks, an image in which luminance and color change, and the like can be considered.

In the present embodiment, the output control unit 354 displays the character image 706 in the reference image (scale 703). Thereby, a subject can be made to pay attention to an appropriate position in the display screen 101.

FIG. 19 is a diagram illustrating another example of the position adjustment image. FIG. 19 is an example of the position adjustment image after a lapse of time from FIG. FIG. 19 shows an example in which the eye position image 805 is displayed larger than the marker 804 because the eye position of the subject approaches the stereo camera 102. In this case, the output control unit 354 may display, for example, the eye position image 805 in a color (for example, red) indicating that the distance is not appropriate. Further, when the eye position image 805 is displayed larger than the marker 804, the output control unit 354 may output a sound such as “much behind”. Further, when the eye position image 805 is displayed smaller than the marker 804, the output control unit 354 may output a sound such as “move forward”.

FIG. 20 is a diagram illustrating another example of the position adjustment image. FIG. 20 is an example of the position adjustment image after a further time has elapsed from FIG. As shown in FIG. 19 and FIG.
The output control unit 354 displays character images 806 and 906 that move within the reference image.

Returning to FIG. By displaying the position adjustment image in step S2, the position of the subject can be adjusted appropriately. As a result, the subsequent processing can be executed with high accuracy. Next, the diagnosis support apparatus 100 executes calibration for eye-gaze detection (eye-gaze detection calibration) (step S3).

In the calibration process, first, the output control unit 354 causes the display unit 210 to display a calibration image (for example, the calibration content selected by the pull-down 611 in FIG. 17).

21 and 22 are diagrams illustrating an example of an image for calibration. First, the output control unit 354 causes the display unit 210 to display a calibration image as shown in FIG. The calibration image includes a circular frame 1001 positioned near the center of the display screen 101 and an object image 1002 positioned near the center of the frame 1001. The output control unit 354 first displays an image as shown in FIG. 21, and then displays the image such that the frame 1001 is gradually reduced toward the center. FIG. 22 is an example of a calibration image displayed after being reduced in this way. The image for calibration after reduction is
A circular frame 1101 and an image 1102 of an object located near the center of the frame 1101 are included. By displaying such an image, the subject can be noted near the center of the display screen 101.

The calibration unit 355 calibrates the calculation parameters for line-of-sight detection so that the line of sight detected by the line-of-sight detection unit 351 faces the center of the display screen 101, assuming that the subject is gazing at the center of the display screen 101. .

Returning to FIG. After the calibration in step S3, the diagnosis support apparatus 100 executes a correction gazing point detection process for correcting the position of the analysis gazing point (step S4). In the correction gazing point detection process, the diagnosis support apparatus 100 detects a gazing point with respect to the mark indicating the target point, that is, a correction gazing point. The correction gazing point detection process will be described later.

Next, the diagnosis support apparatus 100 executes analysis processing using the diagnostic image (step S5).
. In the analysis process, the output control unit 354 displays a diagnostic image, and the viewpoint detection unit 352 detects the gaze point for analysis. In addition, after the correction unit 358 corrects the position of the analysis attention point using the correction attention point, the evaluation unit 359 calculates an evaluation value as an index related to the degree of developmental disorder using the corrected analysis attention point. To do. Details of the analysis process will be described later. It is also possible to support the diagnosis by diagnosing the diagnosis by displaying the result of detection of the gazing point for analysis, as described in FIGS. 50 to 54, for example, without calculating the evaluation value by the evaluation unit 359. It is.

Next, the output control unit 354 outputs the analysis result (evaluation result) by the analysis process to the display unit 210 or the like (step S6). Details of the process of outputting the result will be described later.

Next, details of the correction gazing point detection process in step S4 will be described. FIG. 23 shows the first
It is a flowchart which shows an example of the gaze point detection process for correction | amendment in this embodiment.

In the correction gazing point detection process, first, the output control unit 354 displays the mark SP0 indicating the target point at a predetermined position (origin position) (step S110). Next, the viewpoint detection unit 352 detects the position of the correction gazing point obtained with respect to the mark SP0 (step S).
111). The viewpoint detection unit 352 detects a predetermined number of correction gazing points and calculates their data distribution (step S112).

FIG. 24 is a diagram for explaining a data distribution calculation process. Mark (SP0-SP4)
A determination area centered on is set in advance for each mark. In the present embodiment, a circle with a predetermined radius centered on the mark is used as the determination area. If a predetermined number of correction gaze points cannot be obtained within the judgment area, it is highly likely that the subject did not gaze at the target point mark, and a correction gaze point that is effective for correction is detected. Most likely not. Therefore, in the present embodiment, the viewpoint detection unit 352 detects the correction for detection based on whether or not a predetermined number or more of the gazing points in the determination area are detected during a predetermined time while the landmark is displayed. Judgment is made on whether or not the gaze point is appropriate.

In the data distribution calculation process in step S112, the viewpoint detection unit 352 measures the number of correction gazing points belonging to the determination area. The viewpoint detection unit 352 determines that the detected correction gazing point is appropriate when the number of correction gazing points belonging to the determination area is equal to or greater than a preset threshold (step S113, Yes), the determination-in-area data group excluding the correction gazing point that does not belong to the determination area from among the plurality of correction gazing points detected (step S114).

If it is determined in step S113 that the correction gazing point is not appropriate (No in step S113), the process returns to step S111, and the viewpoint detection unit 352 detects the correction gazing point again.

Next, the viewpoint detection unit 352 calculates the representative value of the in-determination area data group based on the coordinates of each correction gazing point of the in-determination area data group (step S115). In the present embodiment, the average value of the in-determination area data group is calculated as the representative value. The representative value may be a value other than the average value. The representative value may be a standard deviation, for example. In the subsequent processing, the representative value calculated in step S115 is used as the correction gazing point.

Next, when the representative values for all the target point marks have not been calculated (step S116, No), the output control unit 354 displays a mark indicating the next target point (step S117). Then, the process returns to step S111, and representative values for the displayed mark are calculated (steps S111 to S115).

In step S116, when representative values for all target point markers are calculated (step S116, Yes), the correction gazing point detection processing (step S4) is completed.

FIGS. 25-1 to 25-5 are diagrams for explaining the display processing of the mark of each target point in the correction gazing point detection processing (step S4). The output control unit 354 is configured as shown in FIG.
In step S110 shown in FIG. 25, first, as shown in FIG.
Display at 0. Next, in step S117, the output control unit 354 displays the remaining target point marks on the display unit 210 one by one in order.

That is, the output control unit 354, in step S117 as shown in FIG.
The display of SP0 is stopped, SP1 is newly displayed, and step S1 is performed again after calculating the representative value of SP1.
17, the display of SP1 is stopped as shown in FIG. 25-3, and SP2 is newly displayed.
Similarly, the output control unit 354 continues to stop the display of SP2, as shown in FIG.
A new SP3 target point is displayed. Subsequently, as shown in FIG.
3 is stopped and the SP4 target point is newly displayed.

Thus, by displaying each target point one by one in order, it is possible to detect the subject's correction gazing point.

Next, another example of target point marks will be described. 68 to 36 are diagrams showing examples of target point marks. In the example of FIGS. 26 to 36, a mark including a frame having a predetermined shape and an image displayed in the frame is used. In addition, the output control unit 354 displays different images for each of the plurality of target points. Further, the output control unit 354
A moving image in which the display mode of the portion including the center of the frame changes is displayed as the image in the frame.
Thereby, a test subject can be made to gaze at the position nearer the center among the marks of the target points. Note that the portion where the display mode in the frame changes does not have to include the center point of the frame strictly. For example, if you decide what you want to focus on, you don't have to be at the center. In such a case, it is necessary to consider that a point that is not the center is noted when calculating the correction gazing point.

26 to 36, the marks of the five target points are designated as SP0 → SP in FIG.
Displayed in the order of 2 → SP3 → SP4 → SP1. As described above, the display order of the target point marks may be arbitrarily changed.

In FIG. 26, the output control unit 354 first displays a frame 1201 and a character image 1202 located in the frame at a position corresponding to SP0. The image 1202 is a moving image that moves near the center. FIG. 27 shows a state in which an image 1302 of the character after moving so that the nose portion near the center is enlarged is displayed in the frame 1301.

As shown in FIG. 28, the output control unit 354 displays the frame on the target point first,
Thereafter, the character image may be displayed so as to move toward the target point. Thereby, a gaze point can be appropriately guided to the next target point to the subject. In FIG. 28, a frame 1401 is displayed at the target point (the position of SP2 in FIG. 5).
A state in which the character image 1402 is moving toward the frame 1401 is shown.

FIG. 29 shows the state after the character image 1402 in FIG. 28 has moved to the position of the frame 1401.
A state in which an image 1502 of a character different from the image 1402 is displayed is shown. In this way, by displaying different images for each target point, it is possible to focus on the images without getting tired of the subject. FIG. 30 shows a state in which the character image 1602 after moving so that the eye portion near the center of the image 1502 is enlarged is displayed in the frame 1401.

FIG. 31 shows a state where a character image 1702 different from the image 1602 is displayed after the character image 1602 in FIG. 30 has moved to the position of the frame 1701 (the position SP3 in FIG. 5). FIG. 32 shows a state where the character image 1802 after the color near the center of the image 1702 is changed is displayed in the frame 1701.

FIG. 33 shows a state in which a character image 1902 different from the image 1802 is displayed after the character image 1802 in FIG. 32 has moved to the position of the frame 1901 (the position SP4 in FIG. 5). FIG. 34 shows a state in which an image 2002 of the character after moving so that the nose portion near the center of the image 1902 is enlarged is displayed in the frame 1901.

FIG. 35 shows a state in which a character image 2102 different from the image 2002 is displayed after the character image 2002 in FIG. 34 has moved to the position of the frame 2101 (the position SP1 in FIG. 5). FIG. 36 shows a state in which an image 2202 of the character after moving so that the eye portion near the center of the image 2102 is enlarged is displayed in the frame 2101.

Next, details of the analysis processing in step S5 will be described. In the analysis process in step S5, one analysis process or a plurality of analysis processes may be performed.
When performing a plurality of analysis processes, the evaluation unit 359 calculates an evaluation value by integrating the results of the respective analysis processes. FIG. 37 is a flowchart illustrating an example of analysis processing according to the first embodiment.

First, the diagnosis support apparatus 100 executes a gaze point detection process for analysis (step S101).
. In the gazing point detection process for analysis, first, the output control unit 354 displays a diagnostic image on the display unit 21.
Display at 0. Next, the viewpoint detection unit 352 detects the gaze point of the subject when the diagnostic image is displayed on the display unit 210, that is, the gaze point for analysis.

Subsequently, the area dividing unit 356 divides the target area into eight partial areas (A to H) based on the correction gazing point, which is a representative value for the target point mark (step S102).
). Next, the correction unit 358 includes the analysis attention point obtained in step S101.
A partial region in the frame Fe is specified, and further, a partial region in the frame F corresponding to the specified partial region is specified (step S103).

FIG. 38 is a flowchart showing a process of specifying a partial area in the frame Fe in the partial area specifying process (step S103). As shown in FIG. 38, first, the region specifying unit 357 uses the analysis point of interest obtained in step S101 as the viewpoint detection unit 352.
(Step S120).

Next, the area specifying unit 357 determines the coordinates of the gazing point for analysis and the partial areas (A to H on the frame Fe).
) To identify the partial region to which the analysis attention point belongs. Specifically, the area specifying unit 357 specifies the code of the x coordinate of the gaze point for analysis, and then specifies the code of the y coordinate. When the x coordinate of the gazing point for analysis is 0 or more and the y coordinate of the gazing point for analysis is 0 or more (step S121, Yes, step S122, Yes), the region specifying unit 357 further performs analysis. Based on the xy coordinates of the gazing point and the coordinates of the point Fe1, the gazing point for analysis is the origin O and the point Fe1.
It is specified whether it is located above the straight line (straight line O-Fe1) which connects (step S123).
).

When the point of interest for analysis is located on the straight line O-Fe1 or below the straight line O-Fe1 (
In step S123, No), the area specifying unit 357 determines that the analysis gaze point belongs to the first quadrant lower area A (step S124).

When the point of interest for analysis is located above the straight line O-Fe1 (step S123, Ye
s), the area specifying unit 357 determines that the analysis gazing point belongs to the first quadrant upper area B (step S125).

If the x coordinate of the analysis gaze point is 0 or more and the y coordinate of the analysis gaze point is smaller than 0 (step S121, Yes, step S122, No), the region specifying unit 357 further performs the analysis note. Based on the xy coordinates of the viewpoint and the coordinates of the point Fe4, it is specified whether or not the gaze point for analysis is located above the straight line connecting the origin O and the point Fe4 (straight line O-Fe4) (step S126).

When the point of interest for analysis is located on the straight line O-Fe4 or below the straight line O-Fe4 (
In step S126, No), the area specifying unit 357 determines that the analysis gaze point belongs to the lower quadrant area G (step S127).

When the point of interest for analysis is located above the straight line O-Fe4 (step S126, Ye
s), the area specifying unit 357 determines that the analysis gazing point belongs to the upper quadrant H of the fourth quadrant (step S128).

When the x coordinate of the analysis attention point is smaller than 0 and the y coordinate of the analysis attention point is 0 or more (step S121, No, step S129, Yes), the region specifying unit 357 further performs the analysis attention. Based on the xy coordinates of the viewpoint and the coordinates of the point Fe2, it is specified whether or not the gaze point for analysis is located above the straight line connecting the origin O and the point Fe2 (straight line O-Fe2) (step S130).

When the point of interest for analysis is located on the straight line O-Fe2 or below the straight line O-Fe2 (
In step S130, No), the area specifying unit 357 determines that the analysis gaze point belongs to the second quadrant lower area D (step S131).

When the point of interest for analysis is located above the straight line O-Fe2 (step S130, Ye
s), the area specifying unit 357 determines that the analysis gazing point belongs to the upper quadrant C of the second quadrant (step S132).

If the x coordinate of the analysis gaze point is smaller than 0 and the y coordinate of the analysis gaze point is smaller than 0 (step S121, No, step S129, No), the region specifying unit 357 further analyzes the gaze point for analysis. Based on the xy coordinates of and the coordinates of the point Fe3, it is specified whether or not the analysis gazing point is located above a straight line (straight line O-Fe3) connecting the origin O and the point Fe3 (step S).
133).

When the point of interest for analysis is located on the straight line O-Fe3 or below the straight line O-Fe3 (
In step S133, No), the area specifying unit 357 determines that the analysis point of sight belongs to the lower quadrant F of the third quadrant (step S134).

If the point of interest for analysis is located above the straight line O-Fe3 (step S133, Ye)
s), the area specifying unit 357 determines that the analysis gazing point belongs to the third quadrant upper area E (step S135).

This completes the process of specifying the partial region to which the analysis point of interest belongs in step S103.

Returning to FIG. 37, after specifying the partial area in the frame Fe to which the analysis attention point belongs, and further specifying the partial area in the frame F corresponding to the specified partial area, the correction unit 3
58 corrects the position of the gazing point for analysis (step S104). In the analysis gazing point correction process in step S104, the correction unit 358 calculates a corrected position of the analysis gazing point by executing different calculation processes for each partial region identified in step S103. The details of the arithmetic processing are as described above.

Next, the evaluation unit 359 calculates an evaluation value based on the corrected position of the gazing point for analysis (
Step S105). For example, the evaluation unit 359 calculates an evaluation value by integrating a plurality of evaluation items. For example, the evaluation unit 359 calculates an evaluation value based on a ratio in which the position of the gazing point for analysis is included in the evaluation region of the diagnostic image described with reference to FIGS. The evaluation unit 359 may calculate the evaluation value based on the degree of concentration calculated based on the time during which the position of the gazing point for analysis remains in one place. Thus, the analysis process is completed. 51 to 54, when displaying the result to be output to a diagnostician (such as a doctor), the process of calculating the evaluation value in step S105 is omitted, and the viewpoint detection unit is displayed during the display of the diagnostic image. In 352, it is also possible to calculate only in which evaluation area the gaze point of the subject when the diagnostic image is displayed on the display unit 210, that is, the position of the gaze point for analysis is included, or the index of the degree of concentration.

Note that the correction method by the correction unit 358 is an example, and the present invention is not limited to this. Any other method can be applied as long as it is a method for correcting a gaze point for analysis using a gaze point for correction detected from a mark corresponding to a predetermined number (for example, five) target points.

Next, an example of a diagnostic image will be described. 39 to 44 are diagrams illustrating examples of diagnostic images. 39 to 44 show examples of diagnostic images including natural images and geometric images. A geometric image represents an image including one or more geometric patterns. This is because children with developmental disabilities prefer geometric images rather than natural images. The natural image may be a natural object or an image reminiscent of a natural object other than a geometric image. For example, an image (still image or moving image) obtained by capturing a person, an animal, a plant, a natural landscape, or the like with a camera may be used as the natural image. Moreover, you may use the image (still image, moving image) of the character imitating a person, an animal, etc. as a natural image.

For example, the evaluation unit 359 calculates an evaluation value indicating that the degree of developmental disorder is high when a geometric image is watched more than a natural image. For example, the evaluation unit 359 determines whether the viewpoint detected by the viewpoint detection unit is within an evaluation region displaying a geometric pattern or an evaluation region displaying a natural image. For example, the evaluation unit 359 determines whether or not the viewpoint detected by the viewpoint detection unit is within the evaluation region in which the geometric pattern is displayed. When the probability of being within the evaluation area displaying the geometric pattern is high, the evaluation value of the developmental disorder is calculated to be high.

When a diagnostic image is a moving image and the geometric image operates faster than a natural image, and when the color of the geometric image is vivid, the geometric image tends to attract attention. When a diagnostic image is used, even a healthy person may gaze at the geometric image. there,
As shown in the examples of FIGS. 39 to 44, a diagnostic image including a geometric image similar to a natural image may be displayed. For example, color tone, luminance distribution, shape, operation speed, and position in the image (composition)
It is possible to use a geometric image in which at least one of the images is similar or coincident with the natural image. Further, when an object such as a person or an animal is included in the natural image, a geometric image including the same number of geometric patterns as the number of objects may be used. In this case, the geometric pattern corresponding to the person may be a circle or an ellipse.

In addition, when a natural image includes a face such as a person, an animal, and a character imitating a person or an animal, a geometric image including a geometric pattern having the same size as the face may be used. For example, a circle whose difference between the horizontal or vertical length of the face and the radius is a first threshold value or less, an ellipse whose difference between the horizontal length of the face and the short axis length is a second threshold value or less, or the vertical length of the face A geometric image including an ellipse having a difference between the length of the long axis and the length of the major axis equal to or smaller than a third threshold value as a geometric pattern may be used.

Moreover, when using a some diagnostic image, you may use the some diagnostic image from which arrangement | positioning of a natural image and a geometric image differs mutually. For example, after using a diagnostic image including a natural image and a geometric pattern arranged in the first direction with respect to the natural image, the natural image and a second direction different from the first direction with respect to the natural image (for example, A diagnostic image including a geometric pattern arranged in a direction opposite to the first direction may be used. Thereby, the accuracy of diagnosis can be improved.

FIG. 39 is an example of a diagnostic image including a geometric image 2301 and a natural image 2302. In FIG. 39, a natural image 2302 including three persons with a park square and a tree as a background is schematically shown. However, a photograph with such a composition may be used as the natural image 2302. FIG.
Although no color is shown, when a natural color image 2302 is used, a geometric image 2301 including a geometric pattern of a color similar to the color of the natural image 2302 may be used. For example, the background of the geometric pattern may be green in accordance with the green of the background forest or the grass of the square. In addition, a geometric image 2301 including a geometric pattern of a color similar to the color of a person's clothes (in the example of FIG. 39, a portion where a large number of lines overlap) may be used.

FIG. 40 is an example of a diagnostic image including a geometric image 2401 and a natural image 2402. FIG.
In 0, a natural image 2402 including two persons with a park square and trees in the background is schematically shown. FIG. 40 shows an example of a geometric image 2401 including two circular geometric patterns having the same size as the faces of two persons. Although the color is not shown in FIG. 40, for example, a circular geometric pattern having a color that matches or is similar to the color of clothes of two persons may be used.

FIG. 41 is an example of a diagnostic image including a geometric image 2501 and a natural image 2502. FIG.
In FIG. 1, a natural image 2502 including one infant is schematically shown. In addition, FIG. 41 shows an example of a geometric image 2501 including an elliptical geometric pattern having a size similar to the size of the infant's face. Although the color is not shown in FIG. 41, for example, a geometric pattern having the same or similar color as the color of the infant or background in the natural image or a background may be used. Further, the arrangement of colors in the natural image and the arrangement of colors in the geometric pattern may be the same or similar. Further, the arrangement of color shading in the natural image and the shading arrangement of the color of the geometric pattern may be the same or similar.

FIG. 42 is an example of a diagnostic image including a geometric image 2602 and a natural image 2601. FIG.
2 schematically shows a natural image 2601 including four persons against the sky. FIG.
Although no color is shown, a geometric image 2602 including a blue geometric pattern may be used in accordance with the blue sky in the background, for example.

FIG. 43 is an example of a diagnostic image including a geometric image 2702 and a natural image 2701. FIG.
In FIG. 3, a natural image 2701 including two persons cooking is schematically shown. Also, FIG.
3 shows an example of a geometric image 2702 including three circular geometric patterns corresponding to two persons and a circular frying pan. Although the color is not shown in FIG. 43, for example, a geometric image 2702 including a circular geometric pattern having the same or similar color as the color of clothes of the person in front (for example, the color of a line) may be used.

FIG. 44 is an example of a diagnostic image including a geometric image 2802 and a natural image 2801. FIG.
In FIG. 4, a natural image 2801 including three persons with a park square and trees in the background is schematically shown. Although the color is not shown in FIG. 44, for example, the background of the geometric pattern may be green according to the green of the background forest or the grass of the square.

In addition, compared with FIGS. 39-41, in FIGS. 42-44, the arrangement | positioning of a geometric image and a natural image is switched on right and left. Therefore, when using a plurality of diagnostic images, for example, FIG.
Any one of the diagnostic images in FIG. 41 and any one of the diagnostic images in FIGS. 42 to 44 may be displayed in order.

Next, another example of the diagnostic image will be described. 45A and 45B are diagrams illustrating an example of a diagnostic image. FIG. 45A illustrates an example of a diagnostic image including a first object 4501 and a second object 4502 that are moving images in which the shape of a person is represented by dots 4511. The first object 4502 is an image in which the shape of a person turned upside down (rotated 180 degrees) is represented by dots. The second object 4502 is not limited to the upside down direction, and may be a moving image that is rotated in a direction other than the correct direction. As the second object 4502, a moving image that is obtained by rotating the moving image 4501 in reverse (moving in the reverse order) and rotating it may be used. This is because a healthy person evaluates the degree of developmental disability by utilizing the tendency that a moving image in which the direction of the shape of the person is correct is watched. For example, the evaluation unit 359 calculates a lower evaluation value as an index related to the degree of developmental disability when a moving image in which the orientation of the person is correct is watched. In the drawing, a dotted line 4512 connecting the dots 4511 is drawn.
2 is not actually displayed.

The output control unit 354 outputs sound to the speaker 105 and causes the display unit 210 to display a first object 4501 that is a moving image simulating the shape of a person who operates according to the sound, and does not match music. The display unit 210 displays a second object 4502 that is a moving image simulating the shape of the person performing the upside down. The output control unit 354 may display the second object 4502 on the display unit 210 as an object that operates in the reverse order of the first object 4501 and rotates the first object 4501 by 180 degrees. The second object 4502 may play music such as music well known to infants from the speaker 105 and display a moving image in which the shape of a person dancing to the music is represented by dots 4511. You may display the moving image showing the operation of radio exercises with the music of radio exercises. Accordingly, it is possible to configure a moving image in a correct direction similar to familiar music and movement so that a healthy person can more easily watch.

The analysis may be performed using a plurality of diagnostic images as shown in FIG. In this case, a plurality of diagnostic images having differently arranged moving images may be used. For example, after using a diagnostic image including a moving image in the correct direction and a moving image in the opposite direction arranged in the first direction with respect to the moving image, the moving image in the correct direction and the moving image A second direction different from the first direction (
For example, you may use the diagnostic image containing the moving image of the reverse direction arrange | positioned in the reverse direction of the 1st direction. Thereby, the accuracy of diagnosis can be improved. The first direction is, for example, the display unit 210.
Turn right towards. For example, when two diagnostic images are used, the first object 4501 imitating the shape of a person who operates according to sound and the first object 4501 that operates according to sound are rotated as shown in FIG. A first object including a second object 4502
After the diagnostic image is displayed on the display unit, a first object 4503 that is a moving image imitating the shape of a person who operates in accordance with the sound as shown in FIG. A second diagnostic image that is arranged in a second direction different from the direction and includes a fourth object 4504 that is a moving image obtained by rotating the third object 4503 is displayed on the display unit 210. The second direction is, for example, left toward the display unit 210. The first diagnostic image and the second diagnostic image may be ones in which objects that operate in accordance with the sound are interchanged. The third object represents the shape of a person with dots, and the dots move as if dancing to music. 4th
The object is an image in which the shape of a person upside down (rotated 180 degrees) is represented by dots, and the movement of the dots is not in music.

Next, an example of an eye catch image displayed before and after displaying a diagnostic image will be described. The eye-catching image represents an image (pre-diagnosis image) that is displayed in order to attract the subject. 46 to 48 are diagrams showing examples of the eye catch image. As shown in FIG. 46, the character's face 3001 is placed at a position to be noted by the subject (the center of the display screen 101 in the example of FIG. 46).
And an eye-catching image including the food 3002 possessed by the character. For example, before displaying a diagnostic image, the output control unit 354 displays an eye catch image as shown in FIG.

The output control unit 354 may display the background of the diagnostic image and the background of the eye catch image in different display modes. For example, the background color of the diagnostic image may be displayed differently from the background color of the eye catch image. Thus, the subject can easily recognize that the image has been switched to a different image. As a result, the subject can be more focused on the display screen 101. Note that the display mode is not limited to color, and may be, for example, color tone and luminance.

The output control unit 354 may display a moving image that operates so as to reduce around a specific position (for example, a position to be watched) as an eye catch image. The output control unit 354 may output sound to the speaker 105 and display an eye catch image that operates in accordance with the sound. For example, a voice calling to the subject such as “Look and Watch” may be output and an eye-catching image may be displayed. Thereby, the subject can be made to pay attention to the specific position on the display screen 101. As a result, the subject's viewpoint can be correctly detected during analysis processing, and the accuracy of analysis (diagnosis) can be improved.

The output control unit 354 may display the eye catch image so that the eye catch image is reduced around the specific position after the eye catch image is operated according to the sound. FIG. 47 shows an example of an eye catch image after operating from FIG. FIG. 48 shows an eye-catching image after reduction (
An example of a face 3201, food 3202) is shown. The output control unit 354 may display only a part of the image in the frame so as to operate according to the sound. For example, character face 3101
May not be operated, and only the food 3102 may be operated. FIG. 47 shows an example of an image in which the character's nose also changes. Thus, if only the part near the center of the image is operated, it is possible to make the subject gaze at a position closer to the center.

FIG. 49 is a diagram illustrating an example of the timing of displaying an eye catch image and outputting sound. FIG.
In the example 9, the output control unit 354 starts outputting the voice calling “Look, see!” Along with the display of the eye-catching image of the large character. The output control unit 354
An image (moving image) that operates with sound can be used as the eye-catching image.
In this case, for example, the eye catch image of FIG. 46 is displayed at the timing described as “display of image E1” of FIG. For example, the eye catch image of FIG. 47 is displayed at the timing described as “display of image E2”.

When the audio output ends, the output control unit 354 starts contraction of the eye catch image.
During the contraction display of the eye-catching image, the output control unit 354 may not output sound (silence). The output control unit 354 continues the contraction display until the size of the eye catch image becomes zero. For example, the eye catch image of FIG. 48 is displayed at the timing described as “display of image E3” of FIG.

Next, details of the result display screen and the process of outputting the results will be described. As described above, for example, when the result display button 502 is pressed on the menu screen, the output control unit 354 causes the display unit 210 to display the result display screen. 51 to 54 are diagrams illustrating an example of a result display screen.

The output control unit 354 may display a different result display screen for each target person who outputs the evaluation result. For example, the output control unit 354 may display a result by a designated result display method among a result display method output to a subject and a result display method output to a diagnostician (such as a doctor). For example, for the subject, the result is displayed by a result display method that displays only the self-evaluation result, and for the diagnostician, the result is displayed by a result display method that displays a plurality of evaluation results for a plurality of subjects. Also good. Thus, convenience can be improved by switching a display method according to a subject.

The designation method for specifying the result display method by which the output control unit 354 displays the result is arbitrary. For example, the designation of the result display method is saved in an external file (such as an INI file), and the saved designation is used. How to refer to determine the result display method, how to add the function to specify the result display method to be displayed to the menu screen, how to switch the specification according to the authority of the target person authenticated by the login authentication function, etc. Applicable.

FIG. 50 is an example of a result display screen output to the subject. As shown in FIG. 50, for the subject, the result may be displayed by a result display method that displays only the evaluation result (analysis result) for the subject. FIG. 50 shows an example in which an evaluation value that is an index related to the degree of developmental disorder calculated by the evaluation unit 359 is displayed as a percentage in the analysis result output field 3301. When the return button 3302 to the basic menu is pressed, for example, a menu screen as shown in FIG. 16 is displayed. The evaluation unit 359 calculates an evaluation value using a plurality of evaluation items. The evaluation unit 359 determines whether to raise or lower the evaluation value for each of the plurality of evaluation items described above, and outputs the result to the analysis result output column 3301 as a final evaluation value. The evaluation unit 359 may give an evaluation value as a point in each evaluation item.

Instead of displaying the evaluation result on the display unit 210 as shown in FIG. 50 or displaying the evaluation result on the display unit 210, the output control unit 354 supplies a recording medium such as paper on which the evaluation result is printed to the printer 324. You may comprise so that it may output.

51 to 54 are examples of result display screens output to the diagnostician. As shown in FIG. 51, for the diagnostician, the result may be displayed by a result display method for displaying a plurality of evaluation results for a plurality of subjects. The result display screen of FIG. 51 includes a button 3401 for returning to the basic menu,
A display button 3402, an information display column 3403, a subject list 3404, an evaluation item list 3405, and a result display column 3406 are included.

When the return button 3401 to the basic menu is pressed, for example, a menu screen as shown in FIG. 16 is displayed. When a gazing point display button 3402 for displaying a gazing point is pressed, the detected gazing point is displayed by the display method selected by the right radio button of the gazing point display button 3402. Examples of the display method of the gazing point that can be selected include “heat map display”, “display with viewpoint movement locus”, “tile display”, and “replay viewpoint movement”.
“Heat map display” is a display method for plotting the visual line of a subject detected at a certain time as shown in FIG. 55 on a screen. When the coordinates overlap, the output control unit 354 displays the color by changing the color from “blue system → green system → yellow system → orange system → red system”. "Display with viewpoint movement trajectory"
FIG. 56 shows a display method in which the line of sight of a subject detected at a certain time is plotted on a screen, and points that are continuous in time are connected by a line. “Tile display” is a display method in which the color of a block in which a gazing point is detected is changed in accordance with the number of gazing points detected in a block having a certain area as shown in FIG. For example, the output control unit 354 changes the color to “
Change the display to "Blue → Green → Yellow → Orange → Red". “Replay viewpoint movement” is a display method that reproduces the movement of the gaze point of the subject with a moving image. For example, as shown in FIG. 58, the gazing point is expressed by a small circle.

The information display column 3403 is a column for displaying information related to the currently selected subject.
In the information display column 3403, for example, the information input in the information input column 613 on the analysis menu screen in FIG. 17 and the analysis date and time are displayed. When the information editing button is pressed, an editing screen (not shown) for editing the displayed information may be displayed.

The subject list 3404 is an area (first area) for displaying subjects in a selectable manner. FIG.
1 shows an example in which the subject corresponding to the first row from the top is selected. The evaluation item list 3405 is an area (second area) that displays the evaluation items used in the analysis in a selectable manner. For example, one or more evaluation items displayed for the subject selected in the subject list 3404 are displayed in the evaluation item list 3405. The evaluation item list 3405 is numbered for each evaluation item (No.), which scene of a series of diagnostic images (moving images) (Scene)
Type of evaluation method (e.g., title), which area of the diagnostic image should be evaluated (e.g.
Type), data acquisition rate (da) indicating the start time of the corresponding evaluation item in the series of diagnostic images (moving images), the measurement time of the corresponding evaluation item, and the rate at which the gazing point is detected in the diagnostic image within the measurement time
ta rate) is associated. In this example, the evaluation item list 3405 includes
Rate at which viewpoints were detected outside the evaluation area (ratio other than Object), number of times viewpoints were detected within the evaluation area of object 1 (number of scan objects 1), number of times viewpoints were detected within the evaluation area of object 2 (Scan object number 2), the rate at which a gazing point was detected in the evaluation area of object 1 (the ratio of Object 1), the ratio at which a viewpoint was detected in the evaluation area of object 1 (Object
2) is displayed.

The types of evaluation methods are, for example, type-D: eye as object 1, mouth as object 2, and type-E: object 1 as the left half of the diagnostic image and object 2 as the right half of the diagnostic image. The diagnosis support apparatus 100 includes a series of diagnostic images (
Which evaluation is to be performed for which playback time of a video) is stored in association with each other, and is displayed as an evaluation list. The ratio at which the viewpoint is detected in the evaluation area of the object 1 or 2 is calculated as follows. For example, the diagnosis support apparatus 100 detects a gazing point 50 times per second. For example, in the case of an evaluation item with a measurement time of 4 seconds, 200 gaze points are detected. The number of times that the gazing point is detected in the diagnostic image is 180 times, the number of times that the gazing point is detected in the evaluation area of the object 1 is 54 times, and the number of times that the gazing point is detected in the evaluation area of the object 2 is 108. In the case of times, (data rate) = 0.900, (ratio other than Object) = 0.100,
Scan Object number 1) = 54, (Scan Object number 2) = 108, and (ratio of Object 1)
= 0.300, (ratio of Object2) = 0.600. The viewpoint detection unit 352 detects at least the viewpoint of the subject while the diagnostic image is displayed.

The result display column 3406 is an area (third area) for displaying the detection result. Result display field 34
In 06, a diagnostic image displayed for the subject selected from the subject list 3404, for example, a detection result for the selected evaluation item among the evaluation items displayed in the evaluation item list 3405 is displayed. When the evaluation item is not selected, the detection results for the plurality of evaluation items may be displayed together. The result display column 3406 in FIG. 51 shows an example in the case where the detection results for a plurality of evaluation items are combined and displayed. In addition, detection results of other evaluation items having the same type as the evaluation item selected in the evaluation item list 3501 may be combined and displayed in the result display column.

As shown in FIG. 51, the result display field 3406 may be configured so that the display form of the evaluation result can be designated from a plurality of display forms. In FIG. 51, “DataRate”, “Type”
An example is shown in which any of the seven types of display modes “-A” to “Type-F” can be specified by a selection tab.

FIG. 52 is an example of a result display screen when “Type-D” is designated. In FIG. 52, a result display field 35 in which the evaluation results of a plurality of evaluation items corresponding to Type-D are graphed.
Eleven examples are shown. As described above, a plurality of evaluation items having a common type may be displayed in the result display column. In the example of the evaluation item list 3501 in FIG.
The type of evaluation item is displayed in the “Type” column.

FIG. 53 is an example of a result display screen when “Type-E” is designated. In FIG. 53, a result display field 36 that graphs the evaluation results of a plurality of evaluation items corresponding to Type-E.
Eleven examples are shown.

FIG. 54 is an example of a result display screen when “Type-F” is designated. In FIG. 54, a result display field 37 in which the evaluation results of a plurality of evaluation items corresponding to Type-F are graphed.
Eleven examples are shown.

The output control unit 354 corresponds to the changed subject and the changed evaluation item when at least one of the subject selected in the subject list and the evaluation item selected in the evaluation item list is changed. You may comprise so that an evaluation result may be displayed on a result display column. In this embodiment, a column for selecting a subject (subject list), a column for selecting an evaluation item (evaluation item list), and a result display column are displayed in one screen. Then, when the selection of the subject and the evaluation item is switched, the evaluation result displayed in the result display column is switched to the evaluation result corresponding to the changed subject and the evaluation item. Thereby, the diagnostician can display a desired evaluation result with a simple operation.

As described above, according to the present embodiment, for example, the following effects can be obtained.
(1) Since an image used for calibration, an image used for detecting a correction gazing point, and a diagnostic image are displayed in this order, each process necessary for diagnosis is smoothly executed to improve the accuracy of diagnosis. Can do.
(2) Since diagnosis can be performed using a diagnostic image including a natural image and a geometric image similar to the natural image, it is possible to reduce the possibility that a healthy person mistakenly looks at the geometric image. Thereby, the accuracy of diagnosis can be improved.
(3) Since the image representing the eye is displayed on the scale indicating the appropriate position, the subject can determine whether his / her position is correct with respect to the camera and adjust it. Further, since the moving image is displayed in the image corresponding to the reference region representing the range of the appropriate position of the eyes, it is possible to make the subject pay attention to the display screen and appropriately adjust the position of the subject. Thereby, the accuracy of diagnosis can be improved.
(4) Since a plurality of different images are used as the images used for detecting the correction gazing point,
It is possible to focus on the image without getting the subject bored. Thereby, the accuracy of diagnosis can be improved.
(5) Since a diagnostic image including a moving image of the shape of a correctly oriented person that operates in accordance with the sound and a moving image of an incorrect orientation is used, it becomes easier for a healthy person to watch the moving image of the correct orientation. . Thereby, the accuracy of diagnosis can be improved.
(6) Since the background of the diagnostic image and the background of the eye-catching image displayed to draw attention to the subject are displayed in different display modes, the subject can easily recognize that the screen has been switched to a different image. . As a result, the subject can be focused on the display screen, and the accuracy of diagnosis can be improved.
(7) After operating in accordance with the sound, the pre-diagnosis image is displayed so that the pre-diagnosis image is displayed on the display unit before displaying the diagnosis image so as to be reduced around the specific position. Just before starting a diagnostic image while pulling, it is possible to focus on a specific position on the display screen while creating a quiet and easy-to-diagnose situation. As a result, the viewpoint of the subject can be correctly detected during the analysis process, and the accuracy of diagnosis can be improved.
(8) The result display screen for displaying the evaluation result can be switched according to the target person who outputs the evaluation result. Thereby, the convenience for the operator can be improved.
(9) A column for selecting a subject, a column for selecting a diagnostic image, and a result display column are displayed in one screen, and the evaluation displayed in the result display column when the selection of the subject and the diagnostic image is switched. The result is switched to the evaluation result corresponding to the changed subject and diagnostic image. Thereby, the diagnostician can display a desired evaluation result with a simple operation, and can improve the convenience of the diagnostician.

DESCRIPTION OF SYMBOLS 100 Diagnosis support apparatus 101 Display screen 102 Stereo camera 105 Speaker 150 Memory | storage part 202 Right camera 203 Infrared LED light source 204 Left camera 205 Infrared LED light source 208 Drive / IF part 210 Display part 300 Control part 351 Gaze detection part 352 View point detection part 353 Distance detection unit 354 Output control unit 355 Calibration unit 356 Region division unit 357 Region specification unit 358 Correction unit 359 Evaluation unit

Claims (5)

A display unit;
A viewpoint detector for detecting a viewpoint of test subjects that put in the display area of the display unit,
An output control unit that causes the display unit to display an image for correcting the position of the viewpoint detected by the viewpoint detection unit;
With
The image includes a first image and a second image,
The output control unit displays the first image at a predetermined position, displays the second image so as to move toward the predetermined position, and the second image is displayed as the first image. A diagnostic support apparatus, characterized in that it is displayed in a display form that attracts more attention than the image . The diagnosis support apparatus according to claim 1 , wherein the display form changes a shape, a color tone, or luminance.   The diagnosis support apparatus according to claim 1, wherein the second image is an image imitating a person or an animal. And viewpoint detection step of detecting the viewpoint of test subjects that put in the display area of the display unit,
An output control step of causing the display unit to display an image for correcting the position of the viewpoint detected by the viewpoint detection step;
Including
The image includes a first image and a second image,
In the output control step, the first image is displayed at a predetermined position, the second image is displayed so as to move toward the predetermined position, and the second image is displayed on the first position. A diagnostic support method, characterized by displaying in a display form that attracts more attention than an image of And viewpoint detection step of detecting the viewpoint of test subjects that put in the display area of the display unit,
An output control step of causing the display unit to display an image for correcting the position of the viewpoint detected by the viewpoint detection step;
To the computer,
The image includes a first image and a second image,
In the output control step, the first image is displayed at a predetermined position, the second image is displayed so as to move toward the predetermined position, and the second image is displayed on the first position. A diagnostic support program for displaying in a display form that attracts more attention than the image of .