JP6003277B2 - Diagnosis support apparatus and diagnosis support method - Google Patents

Description

  The present invention relates to a diagnosis support apparatus and a diagnosis support method.

  Recently, people with developmental disabilities are said to be on the rise. Developmental disorders are known to be detected early and start treatment to reduce symptoms and increase the effect of adapting to society. In Japan, we are aiming for early detection through interviews at the age of 1 and a half. However, there are problems such as shortage of psychiatrists and time-consuming interviews, and the effect is not sufficient. Therefore, there is a need for an objective and efficient diagnosis support device for developmental disorders.

  For early detection of developmental disorders, it is ideal to be able to diagnose at the age of 1 and a half. In addition, it is necessary to consider the use at the time of screening. One characteristic of children with developmental disabilities is that they do not look at the eyes of their opponents. There has been proposed a method for diagnosing developmental disorders by applying a method of detecting a gazing point by photographing a human face with a camera and calculating a corneal reflection and a pupil position.

JP 2011-206542 A JP 2008-125619 A JP 2005-198743 A JP 2007-213377 A

  In Patent Literature 1, an eye area and a mouth area that reliably includes the mouth of the observation target are specified, the number of frames of a moving image in which the gazing point coordinates are detected in the eye area, and the gazing point coordinates are detected in the mouth area. A method for calculating the number of frames of the moving image, the number of frames of the moving image in which the gaze point coordinates are detected outside the eye area, and the total number of frames to be calculated has been presented. A method was sought.

  The present invention has been made in view of the above, and an object thereof is to provide a diagnosis support apparatus and a diagnosis support method capable of improving the accuracy of diagnosis.

In order to solve the above-described problems and achieve the object, the present invention includes a display control unit that displays an image including a subject on a display unit, a division unit that divides the image into a plurality of regions, and the plurality of the plurality of regions. Among the areas, an area detection unit that detects a specific area including an image of a specific element, a viewpoint detection unit that detects a viewpoint of a subject in the image, and determines whether the viewpoint is included in the specific area, A determination unit that determines a degree of developmental disorder of the subject based on a determination result; and a position detection unit that detects a position of an image of the specific element in the image, wherein the specific element is the subject The right eye and the left eye, and the position detection unit detects a right eye position that is a position of the right eye image in the image, and a left eye position that is a position of the image of the left eye in the image, The dividing unit is any two of the above Boundary frequency is to change the division position so as to pass through the center of the eye position and the left eye position, characterized and this. The present invention also provides a display control unit that displays an image including the subject on a display unit, a dividing unit that divides the image into a plurality of regions, and a specific region that includes an image of a specific element among the plurality of regions. A region detecting unit for detecting the subject, a viewpoint detecting unit for detecting the viewpoint of the subject in the image, and determining whether or not the viewpoint is included in the specific region, and the degree of developmental disorder of the subject based on the determination result And a pattern detection unit that detects a movement pattern of the viewpoint between the plurality of regions, and the determination unit includes the movement pattern and a predetermined movement pattern of the viewpoint. The degree of developmental disorder of the subject is determined by comparing with a reference pattern representing

The present invention also provides a display control step for displaying an image including the subject on a display unit, a division step for dividing the image into a plurality of regions, and a specific region including an image of a specific element among the plurality of regions. A region detecting step for detecting the viewpoint, a viewpoint detecting step for detecting the viewpoint of the subject in the image, and an evaluation unit included in the CPU determines whether or not the viewpoint is included in the specific region, and based on the determination result A determination step of determining the degree of developmental disorder of the subject, and a position detection step of detecting the position of the image of the specific element in the image, wherein the specific element includes the right eye and the left eye of the subject The position detecting step detects a right eye position that is a position of the right eye image in the image and a left eye position that is a position of the left eye image in the image; The dividing step is characterized in that the dividing position is changed so that the boundary between any two of the regions passes through the center of the right eye position and the left eye position. The present invention also provides a display control step for displaying an image including the subject on a display unit, a division step for dividing the image into a plurality of regions, and a specific region including an image of a specific element among the plurality of regions. A region detecting step for detecting the viewpoint, a viewpoint detecting step for detecting the viewpoint of the subject in the image, and an evaluation unit included in the CPU determines whether or not the viewpoint is included in the specific region, and based on the determination result A determination step of determining the degree of developmental disorder of the subject, and a pattern detection step of detecting a movement pattern of the viewpoint between a plurality of the regions, wherein the determination step includes the movement pattern and the viewpoint The degree of developmental disorder of the subject is determined by comparing with a reference pattern representing a predetermined movement pattern.

  The diagnosis support apparatus and diagnosis support method according to the present invention have an effect of improving the accuracy of diagnosis.

FIG. 1 is a diagram illustrating an example of an arrangement of a display unit, a stereo camera, and a light source used in the first embodiment. FIG. 2 is a diagram illustrating an outline of functions of the diagnosis support apparatus according to the first embodiment. FIG. 3 is a block diagram illustrating an example of detailed functions of the respective units illustrated in FIG. FIG. 4 is a flowchart illustrating an example of a diagnosis support process according to the first embodiment. FIG. 5 is a diagram illustrating an example of an image displayed on the display unit. FIG. 6 is a diagram illustrating an example of a gazing point detection result. FIG. 7 is a schematic diagram illustrating an example of division of the target image in FIG. FIG. 8 is a diagram illustrating an example of a result of counting the number of gazing point markers included in a block. FIG. 9 is a diagram illustrating an example of a result of counting the number of gazing point markers included in a block. FIG. 10 is a diagram illustrating an example of a gazing point detection result. FIG. 11 is a schematic diagram illustrating an example of division of the target image in FIG. FIG. 12 is a diagram illustrating an example of a result of counting the number of gazing point markers included in a block. FIG. 13 is a diagram illustrating an example of a result of counting the number of gazing point markers included in a block. FIG. 14 is a block diagram illustrating a configuration example of a control unit according to the second embodiment. FIG. 15 is a flowchart illustrating an example of a diagnosis support process according to the second embodiment. FIG. 16 is a schematic diagram illustrating an example of division of the target image in FIG. FIG. 17 is a diagram illustrating an example of a result of counting the number of gazing point markers included in a block. FIG. 18 is a diagram illustrating an example of a result of counting the number of gazing point markers included in a block. FIG. 19 is a schematic diagram illustrating an example of division of the target image in FIG. FIG. 20 is a diagram illustrating an example of a result of counting the number of gazing point markers included in a block. FIG. 21 is a diagram illustrating an example of a result of counting the number of gazing point markers included in a block. FIG. 22 is a block diagram illustrating a configuration example of a control unit according to the third embodiment. FIG. 23 is a flowchart illustrating an example of a diagnosis support process according to the third embodiment. FIG. 24 is a diagram illustrating an example of a target image used in the third embodiment. FIG. 25 is a diagram illustrating an example of a target image used in the third embodiment. FIG. 26 is a diagram illustrating an example of a target image used in the third embodiment.

  Hereinafter, embodiments of a diagnosis support apparatus and a diagnosis support method according to the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

(First embodiment)
In Patent Document 1 described above, an eye area and a mouth area are defined, and developmental disorders are diagnosed using the number of frames of a moving image in which a gazing point coordinate is detected in each area. However, for example, the eye region is set to be large so that the eyes of the subject (observation subject) are surely included. Therefore, even when the subject does not actually look at the subject's eyes, there is a possibility of erroneously determining that the subject is looking at the eyes. In the method of Patent Document 1, for example, when the subject changes or when the subject moves within the imaging area, the image of the subject taken by the camera may be changed. On the other hand, each area is fixedly defined regardless of the subject. For this reason, there is a possibility that, for example, the position of the eye in the actual image of the subject is out of the eye area by changing the image of the subject. As described above, in the method as disclosed in Patent Document 1, there is a possibility that the accuracy of diagnosis is lowered.

  Therefore, in the first embodiment, an image to be displayed on the display unit is divided into a plurality of regions (blocks), and a block including specific elements (eyes, mouth, etc.) of the subject is specified. Thereafter, it is determined whether or not the subject gazes at the identified block. Thereby, even if it is a case where a subject's image is changed, the block containing a specific element can be specified appropriately, and it can avoid that the precision of a diagnosis falls.

  The first embodiment is an example of diagnosing a developmental disorder of a subject by displaying a diagnostic image (target image) to be observed by the subject on a display unit (monitor). FIG. 1 is a diagram illustrating an example of an arrangement of a display unit, a stereo camera, and a light source used in the first embodiment. As shown in FIG. 1, in the first embodiment, a set of stereo cameras 102 is arranged near the side of the display unit 101. The stereo camera 102 is an imaging unit that can perform stereo shooting with infrared rays, and includes a right camera 202 and a left camera 204.

  Infrared LED (Light Emitting Diode) light sources 203 and 205 are arranged in the circumferential direction immediately before the lenses of the right camera 202 and the left camera 204, respectively. The infrared LED light sources 203 and 205 include an inner peripheral LED and an outer peripheral LED having different wavelengths for emitting light. The pupils of the subject are detected by the infrared LED light sources 203 and 205. As a pupil detection method, for example, the method described in Patent Document 2 can be applied.

  When detecting the line of sight, the space is expressed by coordinates to identify the position. In the present embodiment, the center position of the screen of the display unit 101 is the origin, the top and bottom are the Y coordinate (up is +), the side is the X coordinate (right is +), and the depth is the Z coordinate (front is +). .

  FIG. 2 is a diagram illustrating an outline of functions of the diagnosis support apparatus 100. FIG. 2 shows a part of the configuration shown in FIG. 1 and a configuration used for driving the configuration. As shown in FIG. 2, the diagnosis support apparatus 100 includes a right camera 202, a left camera 204, infrared LED light sources 203 and 205, a speaker 105, a drive / IF unit 208, a control unit 300, and a display unit. 101 and a display unit 210.

  The speaker 105 outputs a sound or the like for prompting the subject to pay attention during calibration.

  The drive / IF unit 208 drives each unit included in the stereo camera 102. The drive / IF unit 208 serves as an interface between each unit included in the stereo camera 102 and the control unit 300.

  The display unit 101 displays a target image. The display unit 210 displays device operations and diagnosis support results.

  FIG. 3 is a block diagram illustrating an example of detailed functions of the respective units illustrated in FIG. As shown in FIG. 3, a display unit 210 and a drive / IF unit 208 are connected to the control unit 300. The drive / IF unit 208 includes camera IFs 314 and 315, an LED drive control unit 316, and a speaker drive unit 322.

  A right camera 202 and a left camera 204 are connected to the drive / IF unit 208 via camera IFs 314 and 315, respectively. The driving / IF unit 208 drives these cameras to image the subject.

  A frame synchronization signal is output from the right camera 202. The frame synchronization signal is input to the left camera 204 and the LED drive control unit 316. This causes the left and right wavelength 1 infrared light sources (wavelength 1-LED 303, wavelength 1-LED 305) to emit light at different timings in the first frame, and correspondingly by the left and right cameras (right camera 202, left camera 204). In the second frame, the left and right wavelength 2 infrared light sources (wavelength 2-LED 304, wavelength 2-LED 306) are caused to emit light in the second frame, and the images from the left and right cameras are captured correspondingly.

  The infrared LED light source 203 includes a wavelength 1-LED 303 and a wavelength 2-LED 304. The infrared LED light source 205 includes a wavelength 1-LED 305 and a wavelength 2-LED 306.

  Wavelength 1-LEDs 303 and 305 emit infrared light having wavelength 1. Wavelength 2-LEDs 304 and 306 irradiate wavelength 2 infrared rays.

  The wavelength 1 and the wavelength 2 are, for example, a wavelength of less than 900 nm and a wavelength of 900 nm or more, respectively. When the reflected light reflected by the pupil is irradiated with infrared light having a wavelength of less than 900 nm, a bright pupil image is obtained as compared with the case where reflected light reflected by the pupil is irradiated with infrared light having a wavelength of 900 nm or more. It is because it is obtained.

  The speaker driving unit 322 drives the speaker 105.

  The control unit 300 controls the entire diagnosis support apparatus 100 and outputs the result to the display unit 210, the speaker 105, and the like. The control unit 300 includes a display control unit 351, a dividing unit 352, an area detection unit 353, a viewpoint detection unit 354, and a determination unit 355.

  The display control unit 351 controls display of various information on the display unit 101 and the display unit 210. For example, the display control unit 351 displays an image of the subject (target image) on the display unit 101. In addition, the display control unit 351 displays the diagnosis result on the display unit 210.

  The dividing unit 352 divides the image displayed on the display unit 210 into a plurality of areas.

  The area detection unit 353 detects a specific area including an image of a specific element among the divided areas. The specific element is, for example, at least one of the subject's eyes and mouth. The specific element is not limited to this. For example, a moving object other than the subject may be used as the specific element.

  The viewpoint detection unit 354 detects the viewpoint of the subject. In this embodiment, the viewpoint detection part 354 detects the viewpoint which is a point which a test subject gazes among the target images displayed on the display part 101 containing the test subject's mother's image etc., for example. As a viewpoint detection method by the viewpoint detection unit 354, any conventionally used method can be applied. Below, similarly to patent document 3, the case where a test subject's viewpoint is detected using the stereo camera 102 is demonstrated to an example.

  In this case, the viewpoint detection unit 354 first detects the gaze direction of the subject from the image captured by the stereo camera 102. The viewpoint detection unit 354 detects the visual line direction of the subject using, for example, the methods described in Patent Documents 1 and 2. Specifically, the viewpoint detection unit 354 obtains the difference between the image captured by irradiating the infrared light having the wavelength 1 and the image captured by irradiating the infrared light having the wavelength 2, and the image in which the pupil image is clarified. Generate. The viewpoint detection unit 354 uses the two images generated as described above from the images captured by the left and right cameras (the right camera 202 and the left camera 204) to determine the position of the pupil of the subject using a stereo vision technique. calculate. In addition, the viewpoint detection unit 354 calculates the position of the corneal reflection of the subject using images captured by the left and right cameras. Then, the viewpoint detection unit 354 calculates a line-of-sight vector representing the line-of-sight direction of the subject from the position of the pupil of the subject and the corneal reflection position.

  The viewpoint detection unit 354 detects, for example, the intersection of the line-of-sight vector represented in the coordinate system as shown in FIG. 1 and the XY plane as the viewpoint of the subject. When the gaze direction of both eyes is obtained, the viewpoint may be measured by obtaining the intersection of the left and right gazes of the subject.

  Note that the method for detecting the viewpoint of the subject is not limited to this. For example, you may detect a test subject's viewpoint by analyzing the image image | photographed using visible light instead of infrared rays.

  The determination unit 355 determines whether or not the viewpoint detected by the viewpoint detection unit 354 is included in the specific region detected by the region detection unit 353. The determination unit 355 determines the degree of developmental disorder of the subject based on the determination result.

  Part or all of the control unit 300 and each unit in the control unit 300 may be realized by hardware, or may be realized by software (program) executed by a CPU or the like.

  A program executed by the diagnosis support apparatus 100 according to the present embodiment is a file in an installable format or an executable format, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk). Recorded on a readable recording medium.

  Further, the program executed by the diagnosis support apparatus 100 of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Further, the program executed by the diagnosis support apparatus 100 of the present embodiment may be configured to be provided or distributed via a network such as the Internet. Further, the program executed by the diagnosis support apparatus 100 of this embodiment may be configured to be provided by being incorporated in advance in a ROM or the like.

  The program executed by the diagnosis support apparatus 100 according to the present embodiment has a module configuration including the above-described units. As actual hardware, a CPU (processor) reads the program from the storage medium and executes it. Each of the above parts is loaded on the main storage device, and each part is generated on the main storage device.

  Next, diagnosis support processing by the diagnosis support apparatus 100 according to the first embodiment configured as described above will be described. FIG. 4 is a flowchart illustrating an example of a diagnosis support process according to the first embodiment. In the present embodiment, the determination criterion is whether or not the rate at which the subject sees the eyes of the person in the target image is greater than or equal to a specified value (for example, 60%).

  First, the display control unit 351 displays a target image including a face image of a person (subject) viewed by the subject on the display unit 101 (step S101). FIG. 5 is a diagram illustrating an example of an image displayed on the display unit 101. FIG. 5 shows an example in which an image including an image 501 of the subject's face is displayed as the target image. The face image 501 includes the eyes 502 of the subject.

  Next, the control unit 300 detects the gaze point coordinates of the subject with respect to the display unit 101 and continues to store the detected coordinates until the gaze point measurement ends (steps S102 to S105).

  Specifically, the viewpoint detection unit 354 calculates the gaze direction of the subject using the relationship between the pupil center and the corneal reflection position (step S102). The viewpoint detection unit 354 detects an intersection (gaze point) between the line-of-sight direction and the display unit 101 (XY plane in FIG. 1) (step S103). The viewpoint detection unit 354 stores the detected coordinate value of the gazing point in a storage unit (not shown) or the like (step S104).

  The controller 300 determines whether or not the measurement is finished (step S105). For example, the control unit 300 determines whether the measurement has been completed, for example, whether a predetermined measurement time has elapsed, or whether a predetermined number of gaze point coordinate values have been obtained. The measurement time (number of times) and the number of coordinate values are determined appropriately in consideration of the age and situation of the subject.

  If the measurement has not been completed (step S105: No), the process returns to step S102 and is repeated. When the measurement is completed (step S105: Yes), the control unit 300 executes a diagnostic process using the measured coordinate values (step S106 to step S111). In addition, you may comprise so that a diagnostic process (step S106-step S111) may also be performed, performing the detection process (step S101-step S105) of a coordinate value. If the detection process and the diagnosis process are separated as shown in FIG. 4, the processing load can be distributed.

  6 to 9 are diagrams illustrating examples of gazing point detection results. The image in FIG. 6 is an example of an image in which a plurality of markers 601 indicating the gaze point of the subject are superimposed on the image in FIG. The display control unit 351 displays an image as shown in FIG. 6 on at least one of the test subject display unit 101 and the result output display unit 210.

  In the diagnosis process, first, the dividing unit 352 divides the entire target image into a plurality of blocks (step S106). In the present embodiment, the dividing unit 352 divides the target image into 23 in the horizontal direction and 33 in the vertical direction. Note that the division method is not limited to this, and any method can be applied as long as the target image is divided into a plurality of regions. For example, the number of divisions is not limited to 23 and 33. Moreover, the shape of the region obtained by the division is not limited to a rectangle (square), and may be an arbitrary shape.

  FIG. 7 is a schematic diagram illustrating an example of division of the target image in FIG. Note that the lower part of FIG. 7 is an enlarged view of a part of blocks (blocks near the eye image) for explanation. A-0 to A-11 and B-0 to B-6 are information for specifying each of the 12 blocks arranged in the horizontal direction and 7 blocks arranged in the vertical direction, and are given for convenience of explanation.

  Returning to FIG. 4, the region detection unit 353 extracts a block (specific region) including human eyes as a specific element from among the divided blocks (step S <b> 107). The area detection unit 353 detects an eye from the image using a generally used image recognition technique such as pattern matching. Then, the region detection unit 353 detects a block including the detected eye as a specific region. Note that the block detection (extraction) method is not limited to this. For example, if the target image to be displayed is known in advance, a method of manually setting a block including eyes may be used in advance.

  Next, the determination unit 355 counts the number of detections of the gazing point for each block (step S108). For example, the determination unit 355 refers to the coordinate value of the gazing point stored in step S104, the coordinate value of each block divided in step S106, and the like, and counts the number of gazing points included in each block. This count value corresponds to the number of detections of the gazing point for each block.

  8 and 9 are diagrams illustrating an example of a result of counting the number of gazing point markers included in each block. In FIG. 9, for the sake of explanation, the background of the block (specific area) containing the eye image is gray.

  Returning to FIG. The determination unit 355 determines whether or not the detected gaze point is included in the specific area. The determination unit 355 determines the degree of developmental disorder of the subject from predetermined rules and determination results. As described above, in the present embodiment, the determination unit 355 determines the degree of developmental disorder based on whether or not the rate at which the subject looks at the eyes of the person in the target image is greater than or equal to a specified value (step S109).

In the example of FIG. 6, the determination is made as follows.
Number of detections of the entire gazing point = 57
Number of times a block containing human eyes is seen = 37
Percentage of people looking at eyes = 37/57 = 64.9%

  Assuming that the specified value is 60%, it is determined that the possibility of developmental disorder is low because the ratio exceeds the specified value. In FIG. 8 (FIG. 9), the number of points of gaze detection is 51. As shown in FIG. 6, since the gazing point is also detected outside the eye periphery, the number of detections of the entire gazing point is 57.

  Next, another determination example is shown in FIGS. The image in FIG. 10 is an example of an image displayed by superimposing a plurality of markers 1001 indicating the gaze point of the subject on the image in FIG. FIG. 11 is a schematic diagram illustrating an example of division of the target image in FIG. 12 and 13 are diagrams illustrating an example of a result of counting the number of gazing point markers included in each block. In FIG. 13, for the sake of explanation, the background of the block (specific area) containing the eye image is gray.

In the example of FIG. 10, the determination is made as follows.
Number of detections of the entire gazing point = 44
Number of times you saw a block containing human eyes = 4
Percentage of people looking at eyes = 4/44 = 9.1%

  Similarly, assuming that the specified value is 60%, it is determined that the possibility of developmental disorder is high because the diagnosis example of FIG.

  As described above, in the present embodiment, it is possible to perform diagnosis support based on a clear quantified determination rule by handling the detected gaze position of the subject in units of blocks. In this embodiment, a block including a specific element (such as an eye or a mouth) of the subject is specified, and the developmental disorder is determined based on whether the subject gazes at the specified block. Thereby, even if it is a case where a subject's image is changed, the block in which a specific element is included can be specified appropriately, and it can avoid that the precision of a diagnosis falls.

(Second Embodiment)
In the first embodiment, the target image is divided into blocks (areas) regardless of the positions of specific elements (eyes, mouth, etc.) in the target image. In this method, for example, the target image may be divided so that only the end of the eye is included in the end of a certain block. For example, assuming that the left eye of the subject is displayed slightly shifted to the left in the example of FIG. 7, the blocks corresponding to A-11 and B-2 (hereinafter referred to as block 11-2), and A-11, B- A block corresponding to 3 (hereinafter referred to as a block 11-3) corresponds to such a block (a block in which an end portion of an eye is included). In this case, the block 11-2 and the block 11-3 are blocks (specific areas) including eyes. Therefore, even if the number of times of gazing at the block 11-2 and the block 11-3 should not be counted as the number of times of gazing at the subject's eyes, there is a possibility that the number of times of gazing is erroneously counted. Can occur.

  Therefore, in the second embodiment, the position of the eye of the subject is detected from the target image, and the target image is divided into a plurality of blocks (areas) based on the detected eye position. As a result, it is possible to reduce the possibility that it is determined that the user has focused on the eyes even though the specific element (such as the eyes) is not being observed. The specific element used as the reference position for division is not limited to the eyes, and may be divided based on the positions of the mouth and other objects.

  FIG. 14 is a block diagram illustrating a configuration example of the control unit 300-2 according to the second embodiment. The functions other than the control unit 300-2 are the same as those in FIG. 2 showing the outline of the functions of the diagnosis support apparatus 100 of the first embodiment, and thus the description thereof is omitted.

  As illustrated in FIG. 14, the control unit 300-2 includes a display control unit 351, a division unit 352-2, a region detection unit 353, a viewpoint detection unit 354, a determination unit 355, and a position detection unit 356-2. And.

  In the second embodiment, the position detector 356-2 is added, and the function of the dividing unit 352-2 is different from that of the first embodiment. Since other configurations and functions are the same as those in FIG. 3 which is a block diagram of the control unit 300 according to the first embodiment, the same reference numerals are given, and description thereof is omitted here.

  The position detection unit 356-2 detects the position (coordinates) of an image of a specific element (such as an eye) in the target image. When the specific element is an eye, the position detection unit 356-2 detects the coordinate values (right eye position and left eye position) of the right eye and left eye of the subject in the target image.

  The division unit 352-2 changes the division position of the block (region) in the target image according to the position detected by the position detection unit 356-2. For example, the dividing unit 352-2 changes the dividing position so that the boundary between any two blocks passes through the approximate center of the right eye position and the left eye position. The approximate center is, for example, a position within a predetermined range from the right eye position and the left eye position center.

  Next, diagnosis support processing by the diagnosis support apparatus according to the second embodiment configured as described above will be described. FIG. 15 is a flowchart illustrating an example of a diagnosis support process according to the second embodiment. In the present embodiment, the determination criterion is whether or not the rate at which the subject sees the eyes of the person in the target image is greater than or equal to a specified value (for example, 60%).

  Steps S201 to S205 are the same processes as steps S101 to S105 in FIG. 4 showing the diagnosis support process of the diagnosis support apparatus 100 according to the first embodiment, and a description thereof will be omitted.

  In the present embodiment, the position detection unit 356-2 obtains the coordinates of the positions of the eyes of the person in the target image (step S206). The position detection unit 356-2 detects an eye from the image by using a generally used image recognition technique such as pattern matching. Then, the position detection unit 356-2 detects the detected center position of the eyes as the coordinates of the positions of the eyes of the person. Note that the eye position detection method is not limited to this. For example, if the target image to be displayed is known in advance, a method of previously setting the eye position manually may be used.

  Next, the dividing unit 352-2 divides the entire target image into a plurality of blocks based on the detected coordinates (step S207). FIG. 16 is a schematic diagram illustrating an example of division of the target image in FIG. As shown in FIG. 16, in the present embodiment, the dividing unit 352-2 divides the square centers having the same height and width as the distance a between the left and right eyes into blocks arranged so as to coincide with the centers of both eyes. To divide. This method makes it possible to capture the relationship between the eye position in the image and the gaze position of the subject more accurately.

  Note that the method of dividing the target image based on the eye position is not limited to the example of FIG. The same effect as in FIG. 7 can be obtained if the method is such that the boundary of at least one block passes through the vicinity of the center position of both eyes. For example, each block shown in FIG. 16 may be further divided vertically into m (m is an integer of 2 or more) and horizontally into n (n is an integer of 2 or more). Moreover, the shape of the region obtained by the division is not limited to a rectangle (square), and may be an arbitrary shape.

  Steps S208 to S212 in FIG. 15 are the same processes as steps S107 to S111 in FIG. 4 showing the diagnosis support process of the diagnosis support apparatus 100 according to the first embodiment, and a description thereof will be omitted.

  Next, a specific example of the diagnosis support process in the second embodiment will be described with reference to FIGS. 17 and 18. 17 and 18 are diagrams illustrating an example of the result of counting the number of gazing point markers included in each block. In FIG. 18, for the sake of explanation, the background of the block (specific region) including the eye image is gray. A-0 to A-5 and B-0 to B-3 are information for specifying each of the six blocks arranged in the horizontal direction and the four blocks arranged in the vertical direction, and are given for convenience of explanation. Hereinafter, a block corresponding to Ax (x is 0 to 5) and By (y is 0 to 3) is represented as a block xy.

The determination unit 355 counts the number of detections of the gazing point for each block. In the examples of FIGS. 17 and 18, the number of detections is as follows.
Block 3-0: 1
Block 1-1: 3
Block 2-1: 15
Block 3-1: 19
Block 2-2: 2
Block 3-2: 1
Other than above: 0

The determination unit 355 determines the degree of developmental disorder based on whether or not the rate at which the subject looks at the eyes of the person in the target image is greater than or equal to a specified value. In the example of FIG. 17, it is determined as follows.
Number of detections of the entire gazing point = 1 + 3 + 15 + 19 + 2 + 1 = 41
Number of times a block containing human eyes is seen = 15 + 19 = 34
Percentage of people looking at eyes = 34/41 = 82.9%

  Assuming that the specified value is 60%, it is determined that the possibility of developmental disorder is low because the ratio exceeds the specified value.

  Next, another example of diagnosis is shown in FIGS. FIG. 19 is a schematic diagram illustrating an example of division of the target image in FIG. 10 described above. 20 and 21 are diagrams illustrating an example of a result of counting the number of gazing point markers included in each block. In FIG. 21, for the sake of explanation, the background of the block (specific area) including the eye image is gray.

In the example of FIGS. 19-21, it determines as follows.
Number of detections of the entire gazing point = 2 + 4 + 2 + 1 + 7 + 5 + 6 + 3 + 1 + 2 + 4 + 4 + 1 = 42
Number of times a block containing human eyes is seen = 5 + 6 = 11
Percentage of people looking at eyes = 11/42 = 26.2%

  Similarly, assuming that the specified value is 60%, it is determined that the possibility of developmental disorder is high because the rate of looking at a person's eyes is significantly below the specified value.

(Third embodiment)
In the embodiment described above, only the frequency of the gazing point is used as a developmental disorder determination rule. The decision rule is not limited to this. Any rule may be used as long as it is a rule that is determined based on the determination result of whether or not the specific area includes a gazing point. For example, a determination rule that determines the block movement pattern of the gazing point by comparing the gazing point positions in time series may be used. In the third embodiment, an example will be described in which a developmental disorder is determined using a pattern in which a gazing point moves a block.

  FIG. 22 is a block diagram illustrating a configuration example of the control unit 300-3 according to the third embodiment. The functions other than the control unit 300-3 are the same as those in FIG. 2 showing the outline of the functions of the diagnosis support apparatus 100 according to the first embodiment, and thus the description thereof is omitted.

  As illustrated in FIG. 22, the control unit 300-3 includes a display control unit 351, a dividing unit 352, a viewpoint detection unit 354, a determination unit 355-3, and a pattern detection unit 357-3. .

  In the third embodiment, the area detection unit 353 is deleted, the pattern detection unit 357-3 is added, and the function of the determination unit 355-3 is different from that of the first embodiment. Since other configurations and functions are the same as those in FIG. 3 which is a block diagram of the control unit 300 according to the first embodiment, the same reference numerals are given, and description thereof is omitted here.

  The pattern detection unit 357-3 detects a pattern (movement pattern) in which the subject's viewpoint moves between a plurality of blocks. The pattern detection unit 357-3 compares, for example, the coordinate values of the divided blocks in the coordinate system of FIG. 1 with the coordinate values of the viewpoint detected by the viewpoint detection unit 354 in the coordinate system of FIG. By doing so, the movement pattern of the viewpoint can be detected.

  The determination unit 355-3 determines the degree of developmental disorder of the subject by comparing the detected movement pattern with the reference pattern. The reference pattern represents a predetermined movement pattern of the viewpoint. For example, a pattern that moves to a specific area from a block other than a block (specific area) including a specific element (eyes, mouth, etc.) can be used as the reference pattern. As described above, one of the characteristics of a developmentally disabled person is that he / she does not look at the eyes of the other party. Therefore, when it matches the reference pattern for moving from a block other than the specific area to the specific area, the determination unit 355-3 determines that the degree of developmental disorder is low.

  As the reference pattern, any one of a pattern representing movement of the viewpoint of the healthy person and a pattern representing movement of the viewpoint of the developmentally disabled person may be used. When the movement pattern matches the former reference pattern, the determination unit 355-3 may determine that the degree of developmental disorder is low. Conversely, when the movement pattern matches the latter reference pattern, the determination unit 355-3 may determine that the degree of developmental disorder is high.

  Further, the reference pattern is not limited to the viewpoint movement pattern between two blocks. A viewpoint movement pattern between three or more blocks may be used as a reference pattern. Further, a reference pattern determined according to a pattern (operation pattern) in which the target operation part operates may be used.

  For example, people with developmental disabilities look at the eyes when the subject's eyes move, but tend to look away from the eyes and look at other parts. Further, people with developmental disabilities tend to gaze at the mouth when the mouth moves, but rarely gaze at the eyes again. Therefore, for example, the criterion “when the target image of the motion pattern in which the mouth moves is displayed, the viewpoint moves to the mouth block, and further, the viewpoint moves to the block including the eyes within a predetermined time after the movement”. A pattern may be used. The determination unit 355-3 may determine that the degree of developmental disorder is low when the detected movement pattern matches this reference pattern.

  Also, a reference pattern including the timing of viewpoint movement with respect to the operation pattern may be used. For example, after the subject's eyes have moved, if the viewpoint moves to a block containing the eyes by the predetermined time, the degree of developmental disorder is greater than if the viewpoint moves to a block containing the eyes after the predetermined time has elapsed You may comprise so that it may determine with (or small).

  Next, diagnosis support processing by the diagnosis support apparatus according to the third embodiment configured as described above will be described. FIG. 23 is a flowchart illustrating an example of a diagnosis support process according to the third embodiment. FIG. 23 illustrates an example in which the possibility of developmental disorder is determined using a reference pattern in which the subject's point of interest moves between the eyes and mouth of a person in the target image. That is, the subject is presented with three types of images, and the point of gaze position while these images are viewed is detected for each block in the same manner as in the first embodiment. Then, the evaluation is performed using the movement pattern of the gazing point position detected separately for the human eyes, mouth, and other blocks. 24 to 26 are diagrams illustrating examples of target images used in this example.

  First, the display control unit 351 displays the target image 1 of FIG. 24 on the display unit 101 (step S301). The target image 1 includes eyes 1402 and a mouth 1401. The target image 1 is an image in which the mouth 1401 is not moving.

  Steps S302, S303, and S304 in FIG. 23 are the same processes as steps S102, S103, and S106 in FIG. 4 showing the diagnosis support process of the diagnosis support apparatus 100 according to the first embodiment, respectively. The description is omitted. Step S307, step S308, and step S309, and step S312, step S313, and step S314, which will be described later, are also the same processing as step S102, step S103, and step S106 in FIG. Omitted.

  The determination unit 355-3 determines whether or not the detected position of the gazing point exists within the block including the eye of the target image (step S305). If the position of the gazing point does not exist in the block including the eyes (step S305: No), the developmental support cannot be determined, and the diagnosis support process is terminated (step S316).

  When the position of the gazing point exists in the block including the eyes (step S305: Yes), the display control unit 351 displays the target image 2 of FIG. 25 on the display unit 101 (step S306). The target image 2 in FIG. 25 includes the eye 1502 and the mouth 1501. Unlike FIG. 24 and FIG. 26, the target image 2 is an image in which the mouth 1501 is moving. Accordingly, the subject's point of gaze position generally moves to a block including the mouth 1501.

  The determination unit 355-3 determines whether or not the detected position of the gazing point exists within the block including the mouth of the target image (step S310). If the position of the gazing point does not exist within the block including the mouth (step S310: No), the development support disorder cannot be determined, and the diagnosis support process is terminated (step S316).

  When the position of the gazing point exists in the block including the mouth (step S310: Yes), the display control unit 351 displays the target image 3 of FIG. 26 on the display unit 101 (step S311). The target image 3 includes eyes 1602 and a mouth 1601. The target image 3 is an image in which the mouth 1601 does not move as in FIG.

  The determination unit 355-3 determines whether or not the detected position of the gazing point exists in a block including the eye of the target image (step S315). When the position of the gazing point exists in the block including the eyes (step S315: Yes), the determination unit 355-3 determines that the possibility of failure is low (step S317). When the position of the gazing point does not exist in the block including the eyes (step S317: No), the determination unit 355-3 determines that the possibility of failure is high (step S318). For example, when the position of the gazing point does not move from the block including the person's mouth, the determination unit 355-3 determines that the possibility of failure is high.

  Thus, in the third embodiment, it is possible to determine a developmental disorder using a pattern in which a gazing point moves between blocks. Note that the method of using a pattern as in the third embodiment can also be applied to the second embodiment.

As described above, according to the first to third embodiments, for example, the following effects can be obtained.
(1) Even when the subject's image is changed, a block including a specific element (eyes, mouth, etc.) can be appropriately specified, and a decrease in the accuracy of diagnosis can be avoided.
(2) The target image is divided into a plurality of blocks on the basis of the positions of specific elements (eyes, mouth, etc.) detected from the target image. For this reason, a block can be divided | segmented in a more suitable position, and the precision of a diagnosis can be improved.
(3) A highly accurate diagnosis can be realized by determining not only the position of the viewpoint but also the movement of the viewpoint.

DESCRIPTION OF SYMBOLS 100 Diagnosis support apparatus 101 Display part 102 Stereo camera 105 Speaker 202 Right camera 203, 205 Infrared LED light source 204 Left camera 208 Drive / IF part 210 Display part 300, 300-2, 300-3 Control part 351 Display control part 352, 352-2 Division unit 353 Region detection unit 354 View point detection unit 355, 355-3 Determination unit 356-2 Position detection unit 357-3 Pattern detection unit

Claims (7)

A display control unit that displays an image including the subject on the display unit;
A dividing unit for dividing the image into a plurality of regions;
An area detection unit for detecting a specific area including an image of a specific element among the plurality of areas;
A viewpoint detection unit for detecting the viewpoint of the subject in the image;
A determination unit that determines whether or not the viewpoint is included in the specific region, and determines a degree of developmental disorder of the subject based on a determination result;
A position detection unit for detecting the position of the image of the specific element in the image;
With
The specific elements are a right eye and a left eye of the subject,
The position detection unit detects a right eye position that is a position of the image of the right eye in the image, and a left eye position that is a position of the image of the left eye in the image;
The dividing unit changes the dividing position so that a boundary between any two of the regions passes through the center of the right eye position and the left eye position.
A diagnostic support apparatus characterized by that. The dividing unit further divides the image into a plurality of the regions such that a width of the region is an interval between the right eye position and the left eye position.
The diagnosis support apparatus according to claim 1. The determination unit determines the degree of developmental disorder of the subject based on the frequency with which the viewpoint is included in the specific region;
The diagnosis support apparatus according to claim 1. A display control unit that displays an image including the subject on the display unit;
A dividing unit for dividing the image into a plurality of regions;
An area detection unit for detecting a specific area including an image of a specific element among the plurality of areas;
A viewpoint detection unit for detecting the viewpoint of the subject in the image;
A determination unit that determines whether or not the viewpoint is included in the specific region, and determines a degree of developmental disorder of the subject based on a determination result;
A pattern detection unit for detecting a movement pattern of the viewpoint between a plurality of the regions;
With
The determination unit determines the degree of developmental disorder of the subject by comparing the movement pattern with a reference pattern representing a predetermined movement pattern of the viewpoint.
A diagnostic support apparatus characterized by that. The specific element is at least one of the subject's eyes and the subject's mouth,
The diagnosis support apparatus according to claim 4, wherein: A display control step of displaying an image including the subject on the display unit;
A dividing step of dividing the image into a plurality of regions;
An area detecting step of detecting a specific area including an image of a specific element among the plurality of areas;
A viewpoint detection step of detecting the viewpoint of the subject in the image;
A determination step in which an evaluation unit included in the CPU determines whether or not the viewpoint is included in the specific region, and determines a degree of developmental disorder of the subject based on a determination result;
A position detecting step of detecting a position of the image of the specific element in the image;
Including
The specific elements are a right eye and a left eye of the subject,
The position detecting step detects a right eye position that is a position of the image of the right eye in the image and a left eye position that is a position of the image of the left eye in the image;
The division step changes the division position so that the boundary between any two of the regions passes through the center of the right eye position and the left eye position.
A diagnostic support method characterized by the above. A display control step of displaying an image including the subject on the display unit;
A dividing step of dividing the image into a plurality of regions;
An area detecting step of detecting a specific area including an image of a specific element among the plurality of areas;
A viewpoint detection step of detecting the viewpoint of the subject in the image;
A determination step in which an evaluation unit included in the CPU determines whether or not the viewpoint is included in the specific region, and determines a degree of developmental disorder of the subject based on a determination result;
A pattern detection step of detecting a movement pattern of the viewpoint between the plurality of regions;
Including
The determination step determines the degree of developmental disorder of the subject by comparing the movement pattern with a reference pattern representing a predetermined movement pattern of the viewpoint.
A diagnostic support method characterized by the above.

Images