JP7074221B2 - Evaluation device, evaluation method, and evaluation program - Google Patents

Description

The present invention relates to an evaluation device, an evaluation method, and an evaluation program.

The corneal reflex method is known as one of the line-of-sight detection techniques. In the corneal reflex method, a subject is irradiated with infrared light emitted from a light source, the subject's eye irradiated with infrared light is photographed with a camera, and the pupil of the pupil with respect to the corneal reflex image, which is a reflection image of the light source on the corneal surface. The position is detected to detect the line of sight of the subject (see, for example, Patent Document 1).

In recent years, various evaluations have been performed using the results of detecting the line of sight of the subject. For example, people with developmental disabilities tend to have a more biased focus on geometric patterns than on human images. For this reason, the subject is evaluated by arranging a person's image and a geometric pattern side by side and detecting which one is more closely watched.

Japanese Unexamined Patent Publication No. 2011-206542

By the way, it is known that the momentary movement of the line of sight of a human is difficult to be manipulated by the person's consciousness, and the movement speed and gaze time simply reflect the psychology of the subject himself / herself. .. Therefore, it is required to evaluate the essential object of interest by deep psychology with high accuracy by detecting the momentary movement of the line of sight of the subject.

The present invention has been made in view of the above, and an object of the present invention is to provide an evaluation device, an evaluation method, and an evaluation program capable of evaluating a subject with high accuracy.

The evaluation device according to the present invention has an image data acquisition unit that acquires image data of the subject's eyeball, a gaze point detection unit that detects position data of the gaze point of the subject based on the image data, and a target image. The gazing point exists in the specific area based on the display control unit to be displayed on the display screen, the area setting unit for setting the specific area corresponding to the target image on the display screen, and the position data of the gazing point. A determination unit that determines whether or not the target image is displayed and outputs the determination data, and based on the determination data, the gaze point is placed in the specific area by the time when a predetermined time elapses after the target image is displayed on the display screen. A calculation unit that calculates time data indicating an existing existence time, an evaluation unit that obtains evaluation data of the subject based on the time data, and an output control unit that outputs the evaluation data are provided, and the predetermined time is The first time from the display of the target image on the display screen to the end of the display of the target image, and the time after the target image is displayed on the display screen, from the first time. Including a short second time, the calculation unit determines the existence time of the gazing point from the display of the target image on the display screen to the elapse of the first time based on the determination data. The first time data to be shown and the second time data showing the existence time of the gazing point from the display of the target image on the display screen to the lapse of the second time are calculated, and the evaluation unit is used. Obtains the evaluation data of the subject based on the first time data and the second time data.

The evaluation method according to the present invention is to acquire image data of the eyeball of the subject, detect the position data of the gazing point of the subject based on the image data, and display the target image on the display screen. Then, based on the setting of the specific area corresponding to the target image on the display screen and the position data of the gazing point, it is determined whether or not the gazing point exists in the specific area, and the determination data is obtained. Based on the output and the determination data, the existence time of the gazing point from the display of the target image on the display screen to the lapse of the first time from the end of the display of the target image is shown. To calculate the first time data and the second time data indicating the existence time of the gazing point from the display of the target image on the display screen to the lapse of the second time shorter than the first time. And, based on the first time data and the second time data, it is included to obtain evaluation data showing the target image that the subject is most psychologically interested in.

The evaluation program according to the present invention has a process of acquiring image data of the eyeball of the subject, a process of detecting the position data of the gazing point of the subject based on the image data, and a process of displaying the target image on the display screen. Based on the process of setting a specific area corresponding to the target image on the display screen and the position data of the gazing point, it is determined whether or not the gazing point exists in the specific area, and the determination data is obtained. Based on the output process and the determination data, the existence time of the gazing point until the first time from the display of the target image on the display screen to the end of the display of the target image is shown. Processing to calculate the first time data and the second time data indicating the existence time of the gazing point from the display of the target image on the display screen to the lapse of the second time shorter than the first time. And, the computer is made to execute the process of obtaining the evaluation data of the subject based on the first time data and the second time data.

According to the present invention, it is possible to perform an evaluation using the gaze detection result of a subject with high accuracy.

FIG. 1 is a perspective view schematically showing an example of a line-of-sight detection device according to the present embodiment. FIG. 2 is a diagram schematically showing the positional relationship between the display device, the stereo camera device, the light source, and the eyes of the subject according to the present embodiment. FIG. 3 is a diagram showing an example of the hardware configuration of the line-of-sight detection device according to the present embodiment. FIG. 4 is a functional block diagram showing an example of the line-of-sight detection device according to the present embodiment. FIG. 5 is a schematic diagram for explaining a method of calculating the position data of the center of curvature of the cornea according to the present embodiment. FIG. 6 is a schematic diagram for explaining a method of calculating the position data of the center of curvature of the cornea according to the present embodiment. FIG. 7 is a flowchart showing an example of the line-of-sight detection method according to the present embodiment. FIG. 8 is a schematic diagram for explaining an example of the calibration process according to the present embodiment. FIG. 9 is a flowchart showing an example of the calibration process according to the present embodiment. FIG. 10 is a schematic diagram for explaining an example of the gaze point detection process according to the present embodiment. FIG. 11 is a flowchart showing an example of the gaze point detection process according to the present embodiment. FIG. 12 is a diagram showing an example of a target image to be displayed on the display device by the display control unit according to the present embodiment. FIG. 13 is a diagram showing an example of a process of setting a reference position. FIG. 14 is a diagram showing an example of the movement of the gazing point of the subject. FIG. 15 is a diagram showing an example of an eye-catching image. FIG. 16 is a diagram showing an example of an eye-catching image. FIG. 17 is a bar graph showing the movement of the gazing point from the display of the target image on the display screen to the elapse of the second time. FIG. 18 is a flowchart showing an example of the evaluation method according to the present embodiment. FIG. 19 is a diagram showing an example of a target image (including the movement of the gazing point of the subject) displayed on the display device by the display control unit according to the present embodiment. FIG. 20 is a flowchart showing an example of the evaluation method according to the present embodiment.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings, but the present invention is not limited thereto. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.

In the following description, a three-dimensional global coordinate system will be set and the positional relationship of each part will be described. The direction parallel to the first axis of the predetermined surface is the X-axis direction, the direction parallel to the second axis of the predetermined surface orthogonal to the first axis is the Y-axis direction, and the directions are orthogonal to each of the first axis and the second axis. The direction parallel to the third axis is the Z-axis direction. The predetermined plane includes an XY plane.
<First Embodiment>
The first embodiment will be described.

FIG. 1 is a perspective view schematically showing an example of the line-of-sight detection device 100 according to the present embodiment. In the present embodiment, the line-of-sight detection device 100 is used as an evaluation device for evaluating an object of interest of a subject.

As shown in FIG. 1, the line-of-sight detection device 100 includes a display device 101, a stereo camera device 102, and a lighting device 103.

The display device 101 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OLED). The display device 101 functions as a display unit.

In the present embodiment, the display screen 101S of the display device 101 is substantially parallel to the XY plane. The X-axis direction is the left-right direction of the display screen 101S, the Y-axis direction is the vertical direction of the display screen 101S, and the Z-axis direction is the depth direction orthogonal to the display screen 101S.

The stereo camera device 102 includes a first camera 102A and a second camera 102B. The stereo camera device 102 is arranged below the display screen 101S of the display device 101. The first camera 102A and the second camera 102B are arranged in the X-axis direction. The first camera 102A is arranged in the −X direction with respect to the second camera 102B. Each of the first camera 102A and the second camera 102B includes an infrared camera, and has, for example, an optical system capable of transmitting near-infrared light having a wavelength of 850 [nm] and an image pickup element capable of receiving the near-infrared light. ..

The lighting device 103 has a first light source 103A and a second light source 103B. The lighting device 103 is arranged below the display screen 101S of the display device 101. The first light source 103A and the second light source 103B are arranged in the X-axis direction. The first light source 103A is arranged in the −X direction with respect to the first camera 102A. The second light source 103B is arranged in the + X direction with respect to the second camera 102B. The first light source 103A and the second light source 103B each include an LED (light emitting diode) light source, and can emit near-infrared light having a wavelength of, for example, 850 [nm]. The first light source 103A and the second light source 103B may be arranged between the first camera 102A and the second camera 102B.

FIG. 2 is a diagram schematically showing the positional relationship between the display device 101, the stereo camera device 102, the lighting device 103, and the subject's eyeball 111 according to the present embodiment.

The lighting device 103 emits near-infrared light, which is the detection light, to illuminate the subject's eyeball 111. The stereo camera device 102 photographs the eyeball 111 with the second camera 102B when the detection light emitted from the first light source 103A irradiates the eyeball 111, and the detection light emitted from the second light source 103B is emitted to the eyeball 111. When irradiated, the eyeball 111 is photographed by the first camera 102A.

A frame synchronization signal is output from at least one of the first camera 102A and the second camera 102B. The first light source 103A and the second light source 103B emit detection light based on the frame synchronization signal. The first camera 102A acquires the image data of the eyeball 111 when the detection light emitted from the second light source 103B irradiates the eyeball 111. The second camera 102B acquires the image data of the eyeball 111 when the detection light emitted from the first light source 103A irradiates the eyeball 111.

When the eyeball 111 is irradiated with the detected light, a part of the detected light is reflected by the pupil 112, and the light from the pupil 112 is incident on the stereo camera device 102. Further, when the eyeball 111 is irradiated with the detection light, a corneal reflex image 113, which is a virtual image of the cornea, is formed on the eyeball 111, and the light from the corneal reflex image 113 is incident on the stereo camera device 102.

By appropriately setting the relative positions of the first camera 102A and the second camera 102B and the first light source 103A and the second light source 103B, the intensity of the light incident on the stereo camera device 102 from the pupil 112 becomes low, and the corneal membrane The intensity of the light incident on the stereo camera device 102 from the reflected image 113 is increased. That is, the image of the pupil 112 acquired by the stereo camera device 102 has low luminance, and the image of the corneal reflex image 113 has high luminance. The stereo camera device 102 can detect the position of the pupil 112 and the position of the corneal reflex image 113 based on the brightness of the acquired image.

FIG. 3 is a diagram showing an example of the hardware configuration of the line-of-sight detection device 100 according to the present embodiment. As shown in FIG. 3, the line-of-sight detection device 100 includes a display device 101, a stereo camera device 102, a lighting device 103, a computer system 20, an input / output interface device 30, a drive circuit 40, and an output device 50. The input device 60 and the audio output device 70 are provided. The computer system 20 includes an arithmetic processing device 20A and a storage device 20B.

The computer system 20, the drive circuit 40, the output device 50, the input device 60, and the audio output device 70 communicate data via the input / output interface device 30.

The arithmetic processing unit 20A includes a microprocessor such as a CPU (central processing unit). The storage device 20B includes a memory or storage such as a ROM (read only memory) and a RAM (random access memory). The arithmetic processing apparatus 20A performs arithmetic processing according to the computer program 20C stored in the storage apparatus 20B.

The drive circuit 40 generates a drive signal and outputs it to the display device 101, the stereo camera device 102, and the lighting device 103. Further, the drive circuit 40 supplies the image data of the eyeball 111 acquired by the stereo camera device 102 to the computer system 20 via the input / output interface device 30.

The output device 50 includes a display device such as a flat panel display. The output device 50 may include a printing device. The input device 60 generates input data by being operated. The input device 60 includes a keyboard or mouse for a computer system. The input device 60 may include a touch sensor provided on the display screen of the output device 50 which is a display device. The voice output device 70 includes a speaker, and outputs voice for calling attention to the subject, for example.

In the present embodiment, the display device 101 and the computer system 20 are separate devices. The display device 101 and the computer system 20 may be integrated. For example, when the line-of-sight detection device 100 includes a tablet-type personal computer, the tablet-type computer may be equipped with a computer system 20, an input / output interface device 30, a drive circuit 40, and a display device 101.

FIG. 4 is a functional block diagram showing an example of the line-of-sight detection device 100 according to the present embodiment. As shown in FIG. 4, the input / output interface device 30 has an input / output unit 302. The drive circuit 40 generates a display device drive unit 402 that generates a drive signal for driving the display device 101 and outputs the drive signal to the display device 101, and a first camera that generates a drive signal for driving the first camera 102A. The first camera input / output unit 404A that outputs to 102A, the second camera input / output unit 404B that generates a drive signal for driving the second camera 102B and outputs it to the second camera 102B, the first light source 103A, and the first light source 103A. It has a light source driving unit 406 that generates a drive signal for driving the two light source 103B and outputs the drive signal to the first light source 103A and the second light source 103B. Further, the first camera input / output unit 404A supplies the image data of the eyeball 111 acquired by the first camera 102A to the computer system 20 via the input / output unit 302. The second camera input / output unit 404B supplies the image data of the eyeball 111 acquired by the second camera 102B to the computer system 20 via the input / output unit 302.

The computer system 20 controls the line-of-sight detection device 100. The computer system 20 includes a display control unit 202, a light source control unit 204, an image data acquisition unit 206, an input data acquisition unit 208, a position detection unit 210, a curvature center calculation unit 212, and a gaze point detection unit 214. It has an area setting unit 216, a determination unit 218, a calculation unit 220, a storage unit 222, an evaluation unit 224, and an output control unit 226. The functions of the computer system 20 are exhibited by the arithmetic processing device 20A and the storage device 20B.

The display control unit 202 causes the display device 101 to display a plurality of target images to be shown to the subject. Further, the display control unit 202 causes the display device 101 to display an eye-catching image for locating the gazing point at the reference position of the display device 101.

The light source control unit 204 controls the light source drive unit 406 to control the operating states of the first light source 103A and the second light source 103B. The light source control unit 204 controls the first light source 103A and the second light source 103B so that the first light source 103A and the second light source 103B emit the detected light at different timings.

The image data acquisition unit 206 acquires image data of the subject's eyeball 111 acquired by the stereo camera device 102 including the first camera 102A and the second camera 102B from the stereo camera device 102 via the input / output unit 302.

The input data acquisition unit 208 acquires the input data generated by operating the input device 60 from the input device 60 via the input / output unit 302.

The position detection unit 210 detects the position data at the center of the pupil based on the image data of the eyeball 111 acquired by the image data acquisition unit 206. Further, the position detection unit 210 detects the position data of the corneal reflex center based on the image data of the eyeball 111 acquired by the image data acquisition unit 206. The center of the pupil is the center of the pupil 112. The corneal reflex center is the center of the corneal reflex image 113. The position detection unit 210 detects the position data of the center of the pupil and the position data of the center of the corneal reflex for each of the left and right eyeballs 111 of the subject.

The curvature center calculation unit 212 calculates the position data of the corneal curvature center of the eyeball 111 based on the image data of the eyeball 111 acquired by the image data acquisition unit 206.

The gaze point detection unit 214 detects the position data of the gaze point of the subject based on the image data of the eyeball 111 acquired by the image data acquisition unit 206. In the present embodiment, the position data of the gazing point means the position data of the intersection of the line-of-sight vector of the subject defined by the three-dimensional global coordinate system and the display screen 101S of the display device 101. The gaze point detecting unit 214 detects the line-of-sight vectors of the left and right eyeballs 111 of the subject based on the position data of the center of the pupil and the position data of the center of curvature of the cornea acquired from the image data of the eyeballs 111. After the line-of-sight vector is detected, the gaze point detection unit 214 detects the position data of the gaze point indicating the intersection of the line-of-sight vector and the display screen 101S.

The area setting unit 216 sets a specific area corresponding to each of the plurality of target images on the display screen 101S of the display device 101.

The determination unit 218 determines whether or not the gazing point exists in a specific area based on the position data of the gazing point, and outputs the determination data. The determination unit 218 determines, for example, whether or not the gazing point exists in the specific region at regular time intervals. The fixed time can be, for example, the period of the frame synchronization signal output from the first camera 102A and the second camera 102B (for example, every 50 [msec]).

Based on the determination data of the determination unit 218, the calculation unit 220 obtains area data indicating a specific area in which the gazing point is first detected among the plurality of specific areas. Further, the calculation unit 220 calculates time data indicating the existence time in which the gazing point has existed in the specific area within a predetermined time after the plurality of target images are displayed on the display device 101 based on the determination data of the determination unit 218. do. The line-of-sight detection device 100 has a timer for measuring the elapsed time since the target images M1 and M2 are displayed on the display device 101. Further, the calculation unit 220 counts the number of determinations that it is determined that the gazing point exists for each specific area. The calculation unit 220 has a counter that counts the number of determinations for each specific area.

The evaluation unit 224 obtains the evaluation data of the subject based on at least the area data. The evaluation data is data showing the target video that the subject is most psychologically interested in among the plurality of target videos displayed on the display device 101. The evaluation unit 224 can also obtain evaluation data based on the area data and the time data. In this case, for example, the evaluation data may be obtained by weighting the area data rather than the time data.

The storage unit 222 stores data for evaluating the subject. In the present embodiment, the storage unit 222 stores, for example, the above-mentioned area data, time data, and evaluation data. Further, the storage unit 222 has a process of acquiring image data of the subject's eyeball, a process of detecting position data of the gaze point of the subject based on the image data, and a process of displaying a plurality of target images on the display unit. , Based on the process of setting a specific area corresponding to each of a plurality of target images on the display screen of the display unit and the position data of the gazing point, it is determined whether or not the gazing point exists in the specific area, and the determination is made. The process of outputting data, the process of obtaining area data indicating the specific area in which the gazing point was first detected among a plurality of specific areas based on the judgment data, and the process of obtaining the evaluation data of the subject based on at least the area data. An evaluation program for causing the computer system 20 to execute the required process and the process of outputting the evaluation data is stored.

The output control unit 226 outputs data to at least one of the display device 101, the output device 50, and the audio output device 70. In the present embodiment, the output control unit 226 causes the display device 101 or the output device 50 to display the area data and the time data calculated by the calculation unit 220. Further, the output control unit 226 causes the display device 101 or the output device 50 to display the position data of the gazing points of the left and right eyeballs 111 of the subject. Further, the output control unit 226 causes the display device 101 or the output device 50 to display the evaluation data output from the evaluation unit 224.

Next, an outline of the processing of the curvature center calculation unit 212 according to the present embodiment will be described. The curvature center calculation unit 212 calculates the position data of the corneal curvature center of the eyeball 111 based on the image data of the eyeball 111.

5 and 6 are schematic views for explaining a method of calculating the position data of the corneal curvature center 110 according to the present embodiment. FIG. 5 shows an example in which the eyeball 111 is illuminated by one light source 103C. FIG. 6 shows an example in which the eyeball 111 is illuminated by the first light source 103A and the second light source 103B. First, the example shown in FIG. 5 will be described. The light source 103C is arranged between the first camera 102A and the second camera 102B. The pupil center 112C is the center of the pupil 112. The corneal reflex center 113C is the center of the corneal reflex image 113. In FIG. 5, the pupil center 112C indicates the pupil center when the eyeball 111 is illuminated by one light source 103C. The corneal reflex center 113C indicates the corneal reflex center when the eyeball 111 is illuminated by one light source 103C.

The corneal reflex center 113C exists on a straight line connecting the light source 103C and the corneal curvature center 110. The corneal reflex center 113C is positioned at the midpoint between the corneal surface and the corneal curvature center 110. The corneal radius of curvature 109 is the distance between the corneal surface and the center of corneal curvature 110.

The position data of the corneal reflex center 113C is detected by the stereo camera device 102. The corneal curvature center 110 exists on a straight line connecting the light source 103C and the corneal reflection center 113C. The curvature center calculation unit 212 calculates the position data on the straight line where the distance from the corneal reflection center 113C is a predetermined value as the position data of the corneal curvature center 110. The predetermined value is a value predetermined from a general corneal radius of curvature value or the like, and is stored in the storage unit 222.

Next, an example shown in FIG. 6 will be described. In the present embodiment, the first camera 102A and the second light source 103B, and the second camera 102B and the first light source 103A are symmetrical with respect to a straight line passing through an intermediate position between the first camera 102A and the second camera 102B. It is placed in the position of. It can be considered that the virtual light source 103V exists at an intermediate position between the first camera 102A and the second camera 102B.

The corneal reflex center 121 indicates the corneal reflex center in the image obtained by taking the eyeball 111 with the second camera 102B. The corneal reflex center 122 indicates the corneal reflex center in the image obtained by taking the eyeball 111 with the first camera 102A. The corneal reflex center 124 indicates a corneal reflex center corresponding to the virtual light source 103V.

The position data of the corneal reflex center 124 is calculated based on the position data of the corneal reflex center 121 and the position data of the corneal reflex center 122 acquired by the stereo camera device 102. The stereo camera device 102 detects the position data of the corneal reflex center 121 and the position data of the corneal reflex center 122 in the three-dimensional local coordinate system defined by the stereo camera device 102. The stereo camera device 102 is subjected to camera calibration by the stereo calibration method in advance, and conversion parameters for converting the three-dimensional local coordinate system of the stereo camera device 102 into the three-dimensional global coordinate system are calculated. The conversion parameter is stored in the storage unit 222.

The curvature center calculation unit 212 converts the position data of the corneal reflex center 121 and the position data of the corneal reflex center 122 acquired by the stereo camera device 102 into position data in the three-dimensional global coordinate system using conversion parameters. The curvature center calculation unit 212 calculates the position data of the corneal reflex center 124 in the three-dimensional global coordinate system based on the position data of the corneal reflex center 121 and the position data of the corneal reflex center 122 defined in the three-dimensional global coordinate system. do.

The corneal curvature center 110 exists on a straight line 123 connecting the virtual light source 103V and the corneal reflection center 124. The curvature center calculation unit 212 calculates the position data on the straight line 123 where the distance from the corneal reflection center 124 is a predetermined value as the position data of the corneal curvature center 110. The predetermined value is a value predetermined from a general corneal radius of curvature value or the like, and is stored in the storage unit 222.

In this way, even when there are two light sources, the corneal curvature center 110 is calculated by the same method as when there is one light source.

The corneal radius of curvature 109 is the distance between the corneal surface and the center of corneal curvature 110. Therefore, the corneal curvature radius 109 is calculated by calculating the position data of the corneal surface and the position data of the corneal curvature center 110.
[Gaze detection method]
Next, an example of the line-of-sight detection method according to the present embodiment will be described. FIG. 7 is a flowchart showing an example of the line-of-sight detection method according to the present embodiment. In the present embodiment, the calibration process (step S100) including the calculation process of the position data of the corneal curvature center 110 and the calculation process of the distance data between the pupil center 112C and the corneal curvature center 110, and the gaze point detection process (step S200). ) Is carried out.
(Calibration process)
The calibration process (step S100) will be described. FIG. 8 is a schematic diagram for explaining an example of the calibration process according to the present embodiment. The calibration process includes calculating the position data of the corneal curvature center 110 and calculating the distance 126 between the pupil center 112C and the corneal curvature center 110.

A target position 130 is set for the subject to gaze. The target position 130 is defined in the three-dimensional global coordinate system. In the present embodiment, the target position 130 is set, for example, at the center position of the display screen 101S of the display device 101. The target position 130 may be set at the end position of the display screen 101S.

The display control unit 202 causes the target image to be displayed at the set target position 130. This makes it easier for the subject to gaze at the target position 130.

The straight line 131 is a straight line connecting the virtual light source 103V and the corneal reflection center 113C. The straight line 132 is a straight line connecting the target position 130 and the pupil center 112C. The center of curvature 110 of the cornea is the intersection of the straight line 131 and the straight line 132. The curvature center calculation unit 212 is based on the position data of the virtual light source 103V, the position data of the target position 130, the position data of the pupil center 112C, and the position data of the corneal reflection center 113C, and the position data of the corneal curvature center 110. Can be calculated.

FIG. 9 is a flowchart showing an example of the calibration process (step S100) according to the present embodiment. The output control unit 226 displays the target image on the display screen 101S of the display device 101 (step S101). The subject can gaze at the target position 130 by gazing at the target image.

Next, the light source control unit 204 controls the light source drive unit 406 to emit the detected light from one of the first light source 103A and the second light source 103B (step S102). The stereo camera device 102 photographs the eyes of the subject with the camera of the first camera 102A and the second camera 102B, whichever has the longer distance from the light source that emits the detected light (step S103).

Next, the light source control unit 204 controls the light source drive unit 406 to emit the detected light from the other light source of the first light source 103A and the second light source 103B (step S104). The stereo camera device 102 photographs the eyes of the subject with the camera of the first camera 102A and the second camera 102B, whichever has the longer distance from the light source that emits the detected light (step S105).

The pupil 112 is detected by the stereo camera device 102 as a dark portion, and the corneal reflection image 113 is detected by the stereo camera device 102 as a bright portion. That is, the image of the pupil 112 acquired by the stereo camera device 102 has low luminance, and the image of the corneal reflex image 113 has high luminance. The position detection unit 210 can detect the position data of the pupil 112 and the position data of the corneal reflex image 113 based on the brightness of the acquired image. Further, the position detection unit 210 calculates the position data of the pupil center 112C based on the image data of the pupil 112. Further, the position detection unit 210 calculates the position data of the corneal reflex center 113C based on the image data of the corneal reflex image 113 (step S106).

The position data detected by the stereo camera device 102 is the position data defined by the three-dimensional local coordinate system. The position detection unit 210 coordinates-converts the position data of the pupil center 112C and the position data of the corneal reflection center 113C detected by the stereo camera device 102 by using the conversion parameters stored in the storage unit 222, and is tertiary. The position data of the pupil center 112C and the position data of the corneal reflex center 113C defined by the original global coordinate system are calculated (step S107).

The curvature center calculation unit 212 calculates a straight line 131 connecting the corneal reflection center 113C defined by the global coordinate system and the virtual light source 103V (step S108).

Next, the curvature center calculation unit 212 calculates a straight line 132 connecting the target position 130 defined on the display screen 101S of the display device 101 and the pupil center 112C (step S109). The curvature center calculation unit 212 obtains an intersection of the straight line 131 calculated in step S108 and the straight line 132 calculated in step S109, and sets this intersection as the corneal curvature center 110 (step S110).

The curvature center calculation unit 212 calculates the distance 126 between the pupil center 112C and the corneal curvature center 110 and stores it in the storage unit 222 (step S111). The stored distance is used to calculate the corneal curvature center 110 in the gaze point detection in step S200.
(Gaze point detection process)
Next, the gaze point detection process (step S200) will be described. The gaze point detection process is performed after the calibration process. The gaze point detection unit 214 calculates the gaze vector of the subject and the position data of the gaze point based on the image data of the eye 111.

FIG. 10 is a schematic diagram for explaining an example of the gaze point detection process according to the present embodiment. The gaze point detection process corrects the position of the corneal curvature center 110 by using the distance 126 between the pupil center 112C and the corneal curvature center 110 obtained in the calibration process (step S100), and the corrected corneal curvature center 110. Includes calculating the gaze point using 110 position data.

In FIG. 10, the gazing point 165 indicates a gazing point obtained from the center of corneal curvature calculated using a general radius of curvature value. The gazing point 166 indicates a gazing point obtained from the center of corneal curvature calculated using the distance 126 obtained in the calibration process.

The pupil center 112C indicates the pupil center calculated in the calibration process, and the corneal reflex center 113C indicates the corneal reflex center calculated in the calibration process.

The straight line 173 is a straight line connecting the virtual light source 103V and the corneal reflection center 113C. The corneal curvature center 110 is the position of the corneal curvature center calculated from a general radius of curvature value.

The distance 126 is the distance between the pupil center 112C and the corneal curvature center 110 calculated by the calibration process.

The corneal curvature center 110H indicates the position of the corrected corneal curvature center after the corneal curvature center 110 is corrected using the distance 126.

The corneal curvature center 110H is obtained because the corneal curvature center 110 exists on the straight line 173 and the distance between the pupil center 112C and the corneal curvature center 110 is 126. As a result, the line of sight 177 calculated when a general radius of curvature value is used is corrected to the line of sight 178. Further, the gazing point on the display screen 101S of the display device 101 is corrected from the gazing point 165 to the gazing point 166.

FIG. 11 is a flowchart showing an example of the gaze point detection process (step S200) according to the present embodiment. Since the processing from step S201 to step S207 shown in FIG. 11 is the same as the processing from step S102 to step S108 shown in FIG. 9, the description thereof will be omitted.

The curvature center calculation unit 212 calculates a position on the straight line 173 calculated in step S207 where the distance from the pupil center 112C is equal to the distance 126 obtained by the calibration process as the corneal curvature center 110H (step S208).

The gaze point detecting unit 214 calculates a line-of-sight vector connecting the pupil center 112C and the corneal curvature center 110H (step S209). The line-of-sight vector indicates the line-of-sight direction that the subject is looking at. The gazing point detection unit 214 calculates the position data of the intersection of the line-of-sight vector and the display screen 101S of the display device 101 (step S210). The position data of the intersection of the line-of-sight vector and the display screen 101S of the display device 101 is the position data of the gazing point of the subject on the display screen 101S defined by the three-dimensional global coordinate system.

The gazing point detection unit 214 converts the gazing point position data defined by the three-dimensional global coordinate system into the position data on the display screen 101S of the display device 101 defined by the two-dimensional coordinate system (step S211). As a result, the position data of the gazing point on the display screen 101S of the display device 101 that the subject looks at is calculated.

Next, the evaluation method according to this embodiment will be described. In the present embodiment, the line-of-sight detection device 100 is used, for example, as an evaluation device for evaluating an object of interest of a subject. In the following description, the line-of-sight detection device 100 may be appropriately referred to as an evaluation device 100.

FIG. 12 is a diagram showing an example of a target image to be displayed on the display device 101 by the display control unit 202. As shown in FIG. 12, the display control unit 202 displays, for example, two target images M1 and M2 on the display screen 101S of the display device 101. The display control unit 202 displays the target image M1 and the target image M2 on the display screen 101S, for example, in a state of being separated from each other.

The target image M1 is, for example, an image of a person's face. In FIG. 12, an image of a female face is shown as an example of the target image M1, but the image is not limited to this, and an image of a male face may be used. Further, the target image M2 is an image about a geometric pattern. FIG. 12 shows, as an example, an image of a pattern in which a plurality of star shapes having the same center but different sizes are overlapped as the target image M2, but the image is not limited to this, and other geometric patterns are shown. It may be a video of. Note that FIG. 12 shows a state in which a frame is displayed around the target images M1 and M2, but the present invention is not limited to this, and the frame may not be displayed.

Further, the area setting unit 216 sets the specific areas A and B on the display screen 101S. The area setting unit 216 sets specific areas A and B in the areas corresponding to the target images M1 and M2, respectively. In the example shown in FIG. 12, the area setting unit 216 sets the specific areas A and B to, for example, circular shapes and equal dimensions, and sets them in the inner portions of the target image M1 and the target image M2.

The specific regions A and B do not have to have the same shape and dimensions, and the shapes and dimensions may be different from each other. Further, the specific regions A and B are not limited to a circular shape, and may have other shapes. For example, the specific areas A and B may have shapes along the contours of the target images M1 and M2, respectively. Further, the area setting unit 216 may set the specific areas A and B to the portions protruding outside the target image M1 and the specific area B, respectively. Further, the area setting unit 216 sets the specific area A in the area including the eyes, the nose and the mouth of the face of the person, but is not limited to this.

Further, the area setting unit 216 sets the reference position P0 on the display screen 101S. The area setting unit 216 sets the reference position P0 at a position intermediate between the specific area A and the specific area B, for example. FIG. 13 is a diagram showing an example of a process of setting the reference position P0. FIG. 13 is an example in which three specific areas A0, B0, and C0 are set. As shown in FIG. 13, when the number of specific regions is three or more, the region setting unit 216 uses a position at a distance d substantially equal to the contour portion of each specific region A0, B0, C0 as a reference position P0. Set. The area setting unit 216 may set a position at a distance substantially equal to the center position of each specific area A0, B0, C0 as a reference position P0.

FIG. 14 is a diagram showing an example of the movement of the gazing point of the subject, and is a diagram showing an example of the gazing point displayed on the display device 101 by the output control unit 226. FIG. 14 shows a gazing point when the target images M1 and M2 of FIG. 12 are viewed. The output control unit 226 causes the display device 101 to display the plot point P indicating the position data of the gazing point of the subject. The detection of the position data of the gazing point is performed, for example, in the cycle of the frame synchronization signal output from the first camera 102A and the second camera 102B (for example, every 50 [msec]). The first camera 102A and the second camera 102B take images in synchronization with each other. Therefore, in the display screen 101S, the area where the plot points P are denser indicates that the subject is gazing. Further, the larger the number of plot points P, the longer the time for the subject to gaze at the area.

FIG. 14 shows a case where the target images M1 and M2 are displayed on the display screen 101S from the state where the gazing point of the subject is located at the reference position P0. In this case, the gazing point first moves to the target image M1 side (left side in FIG. 14) and enters the specific area A. After that, the gazing point moves in the specific area A, then moves to the target image M2 side (right side in FIG. 14), and enters the specific area B of the target image M2. In this case, the gazing point moves from the specific area A to the outside of the area and then enters the specific area B. In the example of FIG. 14, for example, when a subject over 3 years old sees the target images M1 and M2, he / she first psychologically shows an interest in the person and shifts the gaze point, and then is also interested in the adjacent geometric pattern. It shows the operation of shifting the gaze point.

In addition, in order to start the gazing point of the subject from the reference position P0 when the target images M1 and M2 are displayed on the display screen 101S, the display control unit 202 displays the eye-catching image on the display screen 101S and then targets the target. The images M1 and M2 are displayed.

15 and 16 are diagrams showing an example of an eye-catching image. In the example shown in FIG. 15, the display control unit 202 displays, for example, an eye-catching image M3 of a cartoon-drawn animal face superimposed on the reference position P0. Then, the display control unit 202 changes the eye-catching image M3 so as to move the line of sight of the animal to attract the attention (gaze point) of the subject. After that, the display control unit 202 reduces the eye-catching image M3 toward the reference position P0.

Further, in the example shown in FIG. 16, for example, a case where the target images M1 and M2 are used as an eye-catching image is shown. In this case, the display control unit 202 reduces the eye-catching image M3 toward the reference position P0 from the state where the target images M1 and M2 are displayed on the display screen 101S.

In this way, the display control unit 202 can attract the gazing point of the subject to the reference position P0 by displaying the eye-catching image on the display screen 101S and then reducing the size toward the reference position P0. The display control unit 202 has described the case where the eye-catching image is reduced toward the reference position P0 as an example, but the present invention is not limited to this, and for example, the eye-catching image is directed toward the reference position P0. It may be displayed in other modes such as moving.

When subjects are not developmentally disabled, they tend to be more interested in portraits than geometric patterns. In this case, as shown in FIG. 14, the plot point P indicating the gazing point tends to move from the reference position P0 to the person's image, and the person's image tends to be gazed for a longer time than the geometric pattern. On the other hand, when the subject is a person with developmental disabilities, he / she tends to be more interested in geometric patterns than in portrait images. In this case, the plot point P indicating the gazing point moves from the reference position P0 to the geometric pattern, and the geometric pattern tends to be gazed for a longer time than the portrait image. Therefore, it is possible to evaluate the subject by displaying a person's image and a geometric pattern side by side and detecting which one is gazed first and which one is gazed for a long time.

In the evaluation device 100 according to the present embodiment, the calculation unit 220 obtains area data indicating a specific area in which the plot point P indicating the gazing point is first detected. For example, the arithmetic unit 220 sets a flag indicating whether or not the gazing point has entered the specific area A or the specific area B, and a first-look variable which is a variable indicating the first-look area. The flag has a value of either 1 indicating that the flag has entered the specific area A or the specific area B, and 0 indicating that the flag has not entered the specific area A or the specific area B. The first-look variable has a value indicating any of "undecided", "region A", and "region B". The calculation unit 220 can obtain area data based on the value of the flag and the value of the first-look variable. The value of the flag is set to 0 when the target images M1 and M2 are displayed on the display screen 101S, and remains 0 when the gazing point is not in either the specific area A or the specific area B. Become. Further, the value of the flag is changed to 1 when the gazing point enters the specific area A or the specific area B. The value of the first-look variable is set to a value indicating "undecided" when the target images M1 and M2 are displayed on the display screen 101S, for example, and the gazing point is not included in either the specific area A or the specific area B. That is, when the value of the flag is 0, the initial variable remains a value indicating "undecided". In addition, the value of the first-look variable is changed to a value indicating "area A" when the specific area where the gaze point first enters is the specific area A, and the specific area where the gaze point first enters is the specific area. If it is a specific area B, the first-look variable is changed to a value indicating "area B". When the value of the first-look variable indicates "region A", the arithmetic unit 220 obtains the specific region A as region data. Further, when the value of the first-look variable indicates "region B", the arithmetic unit 220 obtains the specific region B as region data.

Further, the calculation unit 220 sets the existence time of the plot points P indicating the gazing point in the specific areas A and B within a predetermined time after the target images M1 and M2 are displayed on the display screen 101S of the display device 101, respectively. Calculate the indicated time data. In the present embodiment, the predetermined time includes, for example, a first time until the display of the target images M1 and M2 is completed, and a second time for detecting the psychological preference of the subject. The second hour is shorter than the first hour. Hereinafter, regarding the time data, the time data when the predetermined time is the first hour is referred to as the first time data, and the time data when the predetermined time is the second hour is referred to as the second time data. The calculation unit 220 can detect the passage of the first time and the second time by using the measurement result of the timer.

Further, in the present embodiment, it can be estimated that the more the number of times the gaze point is determined to exist in the determination unit 218, the longer the existence time of the gaze point in the specific area. Therefore, in the present embodiment, the time data can be, for example, the number of times that the determination unit 218 determines that the gazing point exists before the predetermined time elapses for the specific areas A and B. That is, the time data can be the number of plot points P detected in each of the specific regions A and B by the time the predetermined time elapses. The calculation unit 220 can calculate the time data by using the count result of the counter provided in the determination unit 218.

In the present embodiment, when the evaluation unit 224 obtains the evaluation data based on the area data and the time data, it can be performed as follows, for example. Here, for the specific area A, the counter count result from the display of the target images M1 and M2 on the display screen 101S until the first time elapses is set as the counter value CNTA1, and the counter until the second time elapses. The count result is the counter value CNTA2. Further, for the specific area B, the counter count result from the display of the target images M1 and M2 on the display screen 101S until the first time elapses is set as the counter value CNTB1, and the counter count until the second time elapses. The result is a counter value CNTB2. In this case, the evaluation value ANS for obtaining the evaluation data is expressed by, for example, the following equation (1).

ANS = KS + (CNTA1-CNTB1) x K1
＋ (CNTA2-CNTB2) × K2 ・ ・ ・ (1)
However, in the above equation (1), the value of the constant KS is KS = KSA when the area data is the specific area A, that is, when the subject first sees the specific area A, and the area data is the specific area. When B, that is, when the subject first sees the specific area B, KS = KSB. Further, regarding the constants KSA and KSB, specifically, KSA is a positive value, KSB is a negative value, and the absolute values are equal to each other (| KSA | = | KSB |).

Further, the coefficients KS (KSA, KSB), K1 and K2 are set according to the psychological interests of the subject. Further, regarding the relationship between the coefficient K1 and the coefficient K2, if the first time is t1 and the second time is t2, it can be set as in the following equation (2).

K2> K1 × (t1 / t2) ・ ・ ・ (2)
When the relationship of the equation (2) is used, the evaluation data ANS in which the second time data is weighted rather than the first time data can be obtained. That is, the influence of the counter value (CNTA2, CNTB2) of the specific region that the subject gazes at from the display of the target images M1 and M2 on the display screen 101S until the lapse of the second time t2 becomes large.

When each value is set as described above, when the area data is the specific area A, the value of the coefficient KS is KSA which is a positive value. Further, when the first time data of the specific area A is larger than the first time data of the specific area B, the value of (CNTA1-CNTB1) becomes a positive value. Further, when the second time data of the specific area A is larger than the second time data of the specific area B, the value of (CNTA2-CNTB2) becomes a positive value.

On the other hand, when the area data is the specific area B, the value of the coefficient KS is KSB which is a negative value. Further, when the first time data of the specific area B is larger than the first time data of the specific area A, the value of (CNTA1-CNTB1) becomes a negative value. Further, when the second time data of the specific area B is larger than the second time data of the specific area A, the value of (CNTA2-CNTB2) becomes a negative value.

Therefore, it can be evaluated that the larger the value of the evaluation value ANS is, the higher the degree of interest of the subject is in the target image M1 and the lower the target image M2. In this case, it can be evaluated that the subject is unlikely to be a person with a developmental disorder. Further, it can be evaluated that the smaller the evaluation value ANS value, the higher the degree of interest of the subject in the target image M2 and the lower the target image M1. In this case, it can be evaluated that the subject is highly likely to be a person with a developmental disorder.

FIG. 17 is a time chart showing the movement of the gazing point from the display of the target images M1 and M2 on the display screen 101S to the elapse of the second time in the example of the subject shown in FIG. In FIG. 17, the horizontal axis of the time chart is the elapsed time, and the second time is set as 1 second. As shown in FIG. 17, among the specific areas A and B, the specific area that the subject first saw is the specific area A, and the specific area in which the gazing point exists for a long time before the second time elapses is specified. Area A. In this case, as the evaluation value ANS, the value of the constant KS is KSA, and the value of (CNTA2-CNTB2) × K2 is a positive value.

In the present embodiment, when the evaluation unit 224 outputs the evaluation data, the output control unit 226 may use character data such as "It is unlikely that the subject is a person with a developmental disability" or the like, depending on the evaluation data. , Character data or the like of "the subject is likely to be a person with a developmental disability" can be output to the output device 50.

Next, an example of the evaluation method according to the present embodiment will be described with reference to FIG. FIG. 18 is a flowchart showing an example of the evaluation method according to the present embodiment. In the present embodiment, the display control unit 202 starts the reproduction of the target images M1 and M2 after displaying the eye-catching image M3 on the display screen 101S of the display device 101, and the target images M1 and M2 are displayed on the display screen 101S. Is displayed (step S300).

Further, at the same time as starting the display of the target images M1 and M2, the calculation unit 220 resets the timer to 0 and starts the measurement (step S301). Further, the determination unit 218 resets the counter to 0 and starts the measurement (step S302). Further, the calculation unit 220 sets the value of the flag indicating whether or not the gazing point has entered the specific area A or the specific area B to 0 (step S303). Further, the calculation unit 220 sets the first-look variable to a value indicating “undecided” (step S304). Steps S301, S302, S303, and S304 are performed at the same time.

The gazing point detection unit 214 shows the target images M1 and M2 displayed on the display device 101 to the subject, and the subject on the display screen 101S of the display device 101 every predetermined sampling cycle (for example, 50 [msec]). The position data of the gazing point of is detected (step S305).

When the position data is detected (Yes in step S306), the determination unit 218 determines whether or not there is a plot point P indicating the gazing point for each of the specific areas A and B, and outputs the determination data. (Step S307). The calculation unit 220 determines whether the determination data output from the determination unit 218 is the determination data for the specific area A or the determination data for the specific area B (steps S308 and S309).

When the calculation unit 220 determines that the determination data output from the determination unit 218 is the determination data for the specific area A (Yes in step S308), the calculation unit 220 performs the processes of steps S310 to S315 described below. Further, when the calculation unit 220 determines that the determination data output from the determination unit 218 is the determination data for the specific area B (No in step S308, Yes in step S309), steps from step S316 described below. The process of S321 is performed.

First, when the calculation unit 220 determines that the judgment data output from the judgment unit 218 is the judgment data for the specific area A (Yes in step S308), the calculation unit 220 indicates that the gazing point is the specific area A or the specific area A. It is determined whether or not the value of the flag indicating whether or not the region B has been entered has already been changed to 1 (step S310). When it is determined that the flag value has not been changed to 1 (No in step S310), the arithmetic unit 220 determines that the specific area in which the gazing point is first detected is the specific area A (step S311), and sees it for the first time. The variable is changed to a value indicating "region A" (step S312). After that, the calculation unit 220 determines whether or not the second time has elapsed based on the measurement result of the timer (step S313). When the calculation unit 220 determines that the second time has not elapsed (No in step S313), the value of the counter value CNTA2 is set to +1 (step S314). That is, the process of the calculation unit 220 in step S314 is a process of increasing the number of times it is determined that the plot point P exists in the specific area A from the time when the target images M1 and M2 are displayed until the second time elapses. Is. After that, the calculation unit 220 sets the value of the counter value CNTA1 to +1 (step S315). That is, the process of the calculation unit 220 in step S315 is a process of increasing the number of times it is determined that the plot point P exists in the specific area A from the time when the target images M1 and M2 are displayed until the first time elapses. Is. If it is determined in step S313 that the second time has elapsed (Yes in step S313), the calculation unit 220 performs step S315 without performing step S314.

Next, when the calculation unit 220 determines that the determination data output from the determination unit 218 is the determination data for the specific area B (Yes in step S309), the calculation unit 220 performs the same processing as in steps S310 to the specific area S315. Do about B. That is, the calculation unit 220 determines whether or not the flag value indicating whether or not the gazing point has entered the specific area A or the specific area B has already been changed to 1 (step S316). When it is determined that the flag value has not been changed to 1 (No in step S316), the arithmetic unit 220 determines that the specific area where the gaze point is first detected is the specific area B (step S317), and sees it for the first time. The variable is changed to a value indicating "region B" (step S318). After that, the calculation unit 220 determines whether or not the second time has elapsed based on the measurement result of the timer (step S319). When the calculation unit 220 determines that the second time has not elapsed (No in step S319), the value of the counter value CNTB2 is set to +1 (step S320). That is, the process of the calculation unit 220 in step S320 is a process of increasing the number of times it is determined that the plot point P exists in the specific area B from the display of the target images M1 and M2 to the lapse of the second time. Is. After that, the calculation unit 220 sets the value of the counter value CNTB1 to +1 (step S321). That is, the process of the calculation unit 220 in step S321 is a process of increasing the number of times it is determined that the plot point P exists in the specific area B from the time when the target images M1 and M2 are displayed until the first time elapses. Is. If it is determined in step S319 that the second time has elapsed (Yes in step S319), the calculation unit 220 performs step S321 without performing step S320.

After that, the calculation unit 220 determines whether or not the first time has elapsed based on the measurement result of the timer (step S322). When the calculation unit 220 determines that the first time has not elapsed (No in step S322), the calculation unit 220 causes the process after the above step S305 to be repeated. When the calculation unit 220 determines that the first time has elapsed (Yes in step S322), the display control unit 202 stops the reproduction of the target images M1 and M2 and ends the display (step S323).

After finishing the display, the evaluation unit 224 calculates the evaluation value ANS based on the area data and the time data obtained from the above processing result (step S324), and obtains the evaluation data based on the evaluation value ANS. After that, the output control unit 226 outputs the evaluation data obtained by the evaluation unit 224 (step S325).

As described above, the evaluation device 100 according to the present embodiment has an image data acquisition unit 206 that acquires image data of the subject's eyeball, and a gaze point detection that detects the position data of the gaze point of the subject based on the image data. Unit 214, a display control unit 202 for displaying the target images M1 and M2 on the display device 101, and an area setting for setting specific areas A and B corresponding to the target images M1 and M2 on the display screen 101S of the display device 101. Based on the position data of the gaze point and the determination unit 216, it is determined whether or not the gaze point exists in the specific areas A and B, respectively, and the determination unit 218 that outputs the determination data and the specific area based on the determination data. Of A and B, the calculation unit 220 that obtains the area data indicating the specific area where the gazing point is first detected, the evaluation unit 224 that obtains the evaluation data of the subject based on at least the area data, and the output that outputs the evaluation data. It is provided with a control unit 226.

With this configuration, after the target images M1 and M2 are displayed on the display screen 101S, the area data indicating the specific area in which the subject's gaze point is first detected among the specific areas A and B is obtained, and is based on the area data. The subject's evaluation data is requested. Therefore, the evaluation device 100 can evaluate the essential object of interest by deep psychology by the momentary movement of the line of sight of the subject. As a result, the evaluation device 100 can evaluate the subject with high accuracy.

Further, in the evaluation device 100 according to the present embodiment, the calculation unit 220 has the gaze point of the specific area A from the display of the target images M1 and M2 on the display device 101 until a predetermined time elapses based on the determination data. , The time data indicating the existence time existing in B is calculated, and the evaluation unit 224 obtains the evaluation data based on the area data and the time data. As a result, since the types of data used when obtaining the evaluation data are increased, the subject of interest of the subject can be evaluated with higher accuracy.

Further, in the evaluation device 100 according to the present embodiment, the predetermined time is the first time from the display of the target images M1 and M2 on the display device 101 to the end of the display of the target images M1 and M2, and the first time. The calculation unit 220 indicates the existence time of the gazing point from the display of the target images M1 and M2 on the display device 101 until the first time elapses, including the second time shorter than the second time. The evaluation unit 224 calculates the first time data and the second time data indicating the existence time of the gazing point from the display of the target images M1 and M2 on the display device 101 until the second time elapses. Evaluation data is obtained based on the area data, the first time data, and the second time data. As a result, since the types of data used when obtaining the evaluation data are many and detailed, it is possible to evaluate the subject of interest of the subject in more detail and with high accuracy.

Further, in the evaluation device 100 according to the present embodiment, the evaluation unit 224 obtains the evaluation data by weighting the second time data rather than the first time data. As a result, the influence of the time data when the time after the target images M1 and M2 are displayed on the display device 101 is short can be increased, so that the subject's target of interest can be evaluated with higher accuracy.

Further, in the evaluation device 100 according to the present embodiment, the display control unit 202 displays the eye-catching image M3 for positioning the gazing point at the reference position P0 of the display screen 101S, and then refers to the target images M1 and M2. It is displayed at a position distant from each other with respect to the position P0. As a result, the gazing point of the subject can be positioned at the reference position P0 with a higher probability, so that the subject's object of interest can be evaluated in more detail and with high accuracy.

In the present embodiment, the evaluation device 100 determines in the calculation unit 220 whether the value of the first-look variable indicates "region A" or "region B", thereby indicating the intrinsic interest of the subject by the deep psychology. You may simply evaluate whether the target is the target image M1 or the target image M2. In this case, when the value of the first-look variable indicates "region A", the calculation unit 220 can determine that the target image M1 is the essential object of deep psychology of the subject. Further, when the value of the first-look variable indicates "region B", the arithmetic unit 220 can determine that the target image M2 is the essential target of the subject's deep psychology.

<Second Embodiment>
The second embodiment will be described. In the following description, the same or equivalent components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be simplified or omitted. FIG. 19 is a diagram showing an example of a target image to be displayed on the display device 101 by the display control unit 202 according to the present embodiment.

As shown in FIG. 19, the area setting unit 216 sets only the specific area A on the display screen 101S. The area setting unit 216 sets the specific area A in the area corresponding to the target image M1. In the example shown in FIG. 19, the area setting unit 216 sets the specific areas A in a circular shape, for example, and sets them in the inner portion of the target image M1 as in the first embodiment.

In the present embodiment, since the evaluation unit 224 has only one specific area A, the evaluation unit 224 obtains the evaluation data based on the time data. In this case, for example, it can be performed as follows. Here, for the specific area A, the counter count result from the display of the target images M1 and M2 on the display screen 101S until the first time elapses is set as the counter value CNTA1, and the counter until the second time elapses. The count result is the counter value CNTA2. In this case, the evaluation value ANS for obtaining the evaluation data is expressed by, for example, the following equation (3).

ANS = CNTA1 x K1
＋ CNTA2 × K2 ・ ・ ・ (3)
However, in the above formula (3), K1 and K2 are set according to the psychological interests of the subject. Further, regarding the relationship between the coefficient K1 and the coefficient K2, if the first time is t1 and the second time is t2, it can be set as in the following equation (4).

K2> K1 × (t1 / t2) ・ ・ ・ (4)
When the relationship of the equation (4) is used, the evaluation data ANS in which the second time data is weighted rather than the first time data can be obtained. That is, the influence of the counter value CNTA2 in the specific region that the subject gazes at from the time when the target images M1 and M2 are displayed on the display screen 101S until the second time t2 elapses becomes large.

Next, an example of the evaluation method according to the present embodiment will be described with reference to FIG. FIG. 20 is a flowchart showing an example of the evaluation method according to the present embodiment. In the evaluation method according to the present embodiment, the display control unit 202 displays the eye-catching image M3 on the display screen 101S of the display device 101, then starts reproducing the target images M1 and M2, and displays the target image on the display screen 101S. Display M1 and M2 (step S400). Hereinafter, each process from step S401 to step S407 is the same as each process from step S301 to step S307 of the first embodiment.

Next, when the determination data is output from the determination unit 218 in step S407, the calculation unit 220 determines that the determination data is for the specific area A (step S408), and thereafter from step S310 in the first embodiment. The processing of steps S410 to S415, which is the same processing as that of step S315, is performed.

After that, the calculation unit 220 determines whether or not the first time has elapsed based on the measurement result of the timer (step S422). When the calculation unit 220 determines that the first time has not elapsed (No in step S422), the calculation unit 220 causes the process after the above step S405 to be repeated. When the calculation unit 220 determines that the first time has elapsed (Yes in step S422), the display control unit 202 stops the reproduction of the target images M1 and M2 and ends the display (step S423).

After finishing the display, the evaluation unit 224 calculates the evaluation value ANS based on the area data and the time data obtained from the above processing result (step S424), and obtains the evaluation data based on the evaluation value ANS. After that, the output control unit 226 outputs the evaluation data obtained by the evaluation unit 224 (step S425).

As described above, according to the present embodiment, the display control unit 202 displays the eye-catching image M3 for positioning the gazing point at the reference position P0 of the display screen 101S, and then refers to the target images M1 and M2. In addition to displaying the images at positions distant from each other with respect to the position P0, the calculation unit 220 identifies the gazing point from the display of the target images M1 and M2 on the display device 101 until a predetermined time elapses based on the determination data. The time data indicating the existence time existing in the area A is calculated, and the evaluation unit 224 obtains the evaluation data based on the time data. As a result, the detection can be started with the gaze point of the subject positioned at the reference position P0 with a higher probability, and then the evaluation data is obtained based on the time data, so that the subject's object of interest is more accurate. Can be evaluated.

The technical scope of the present invention is not limited to the above-described embodiment, and changes can be made as appropriate without departing from the spirit of the present invention. For example, in the above embodiment, the case where the second time is the time for detecting the psychological preference of the subject has been described as an example, but the present invention is not limited to this. For example, the second time can be set as the time until the total number of plot points P determined by the determination unit 218 reaches a predetermined number. In this case, the plot points P from the display of the target images M1 and M2 until the predetermined number is reached are weighted as compared with the other plot points P. Further, in the above embodiment, two types of time, the first time and the second time, are set as the predetermined time, but the present invention is not limited to this, and the predetermined time may be set by dividing the predetermined time into three or more types of time. good.

A, A0, B, B0, C0 ... specific area, M1, M2 ... target image, P0 ... reference position, P ... plot point, M3 ... eye-catching image, ANS ... evaluation value, CNTA1, CNTA2, CNTB1, CNTB2 ... counter Value, 20 ... computer system, 20A ... arithmetic processing device, 20B ... storage device, 30 ... input / output interface device, 40 ... drive circuit, 50 ... output device, 60 ... input device, 70 ... voice output device, 100 ... line-of-sight detection Device, evaluation device, 101 ... display device, 101S ... display screen, 102 ... stereo camera device, 102A ... first camera, 102B ... second camera, 103 ... lighting device, 103A ... first light source, 103B ... second light source, 103C ... light source, 103V ... virtual light source, 109 ... camera radius of curvature, 110, 110H ... center of corneal curvature, 111 ... eyeball, eye, 112 ... pupil, 112C ... center of pupil, 113 ... corneal reflection image, 113C, 121, 122, 124 ... Corneal reflection center, 123, 131, 132, 173 ... Straight line, 130 ... Target position, 165, 166 ... Gaze point of view, 177, 178 ... Line of sight, 202 ... Display control unit, 204 ... Light source control unit, 206 ... Image data Acquisition unit, 208 ... Input data acquisition unit, 210 ... Position detection unit, 212 ... Curvature center calculation unit, 214 ... Gaze point detection unit, 216 ... Area setting unit, 218 ... Judgment unit, 220 ... Calculation unit, 222 ... Storage unit , 224 ... Evaluation unit, 226 ... Output control unit, 302 ... Input / output unit, 402 ... Display device drive unit, 404A ... First camera input / output unit, 404B ... Second camera input / output unit, 406 ... Light source drive unit

Claims (6)

An image data acquisition unit that acquires image data of the subject's eyeball,
A gaze point detection unit that detects the position data of the gaze point of the subject based on the image data, and
A display control unit that displays multiple target images on the display screen,
An area setting unit that sets a specific area corresponding to one of the plurality of target images on the display screen, and an area setting unit.
A determination unit that determines whether or not the gazing point exists in the specific area based on the position data of the gazing point and outputs determination data.
Based on the determination data, a calculation unit that calculates time data indicating the existence time of the gazing point in the specific area from the time when the target image is displayed on the display screen to the time when a predetermined time elapses.
An evaluation unit that obtains evaluation data of the subject based on the time data,
It is equipped with an output control unit that outputs the evaluation data.
The predetermined time is the first time from the display of the target image on the display screen to the end of the display of the target image, and the time after the target image is displayed on the display screen. Including the second hour, which is shorter than the first hour
Based on the determination data, the calculation unit includes first time data indicating the existence time of the gazing point from the display of the target image on the display screen to the elapse of the first time, and the display. The second time data indicating the existence time of the gazing point from the display of the target image on the screen to the lapse of the second time is calculated.
The evaluation unit is an evaluation device that obtains evaluation data of the subject based on the first time data and the second time data. The evaluation device according to claim 1, wherein the evaluation unit weights the second time data more than the first time data to obtain the evaluation data. The display control unit displays an eye-catching image for positioning the gazing point at a reference position on the display screen, and then displays a plurality of the target images at positions separated from each other with respect to the reference position. The evaluation device according to claim 1 or 2. The evaluation device according to any one of claims 1 to 3, wherein the target image is an image including a person's face. It is an evaluation method performed by an evaluation device.
Acquiring image data of the subject's eyeball and
To detect the position data of the gazing point of the subject based on the image data,
Displaying multiple target images on the display screen and
Setting a specific area corresponding to one of the plurality of target images on the display screen, and
Based on the position data of the gazing point, it is determined whether or not the gazing point exists in the specific area, and the determination data is output.
Based on the determination data, the first time data indicating the existence time of the gazing point until the first time elapses from the display of the target image on the display screen to the end of the display of the target image. To calculate the second time data indicating the existence time of the gazing point from the display of the target image on the display screen to the lapse of the second time shorter than the first time.
An evaluation method including obtaining evaluation data showing the target image of which the subject is psychologically most interested based on the first time data and the second time data. The process of acquiring image data of the subject's eyeball and
A process of detecting the position data of the gazing point of the subject based on the image data, and
Processing to display multiple target images on the display screen,
The process of setting a specific area corresponding to one of the plurality of target images on the display screen, and
A process of determining whether or not the gazing point exists in the specific area based on the position data of the gazing point and outputting the determination data.
Based on the determination data, the first time data indicating the existence time of the gazing point until the first time elapses from the display of the target image on the display screen to the end of the display of the target image. , And the process of calculating the second time data indicating the existence time of the gazing point from the display of the target image on the display screen to the lapse of the second time shorter than the first time.
An evaluation program that causes a computer to execute a process of obtaining evaluation data of the subject based on the first time data and the second time data.

Images