JP5776323B2 - Corneal reflection determination program, corneal reflection determination device, and corneal reflection determination method - Google Patents

Description

  The present invention relates to a corneal reflection determination program and the like.

  There is a technique for detecting a person's line of sight by analyzing an image of a person. The technology for detecting the line of sight is used for operation interfaces of various devices and when determining the state of a person.

  Here, a conventional technique for detecting a person's line of sight will be described. As a technique for detecting a line of sight, there is a corneal reflection method for detecting a line of sight by extracting a human pupil and a corneal reflection region from an image. FIG. 9 is a diagram illustrating an example of a pupil and a corneal reflection region. In FIG. 9, 1a represents a pupil, and 1b represents a corneal reflection region. In the prior art, infrared rays are irradiated and the corneal reflection region 1b is photographed with a camera.

  By the way, when a person wears spectacles, a spectacle lens reflection region having a shape similar to the corneal reflection region may appear. FIG. 11 is a diagram illustrating an example of a cornea reflection region and a spectacle lens reflection region. As shown in FIG. 11, when a person is wearing glasses, the corneal reflection region 6a and the spectacle lens reflection region 6b may be reflected in the image. Thus, when the corneal reflection region 6a and the spectacle lens reflection region 6b are reflected in the image, it cannot be determined which is the true corneal reflection region, and the line of sight cannot be detected accurately.

  In order to solve this problem, in the prior art, there is a technique of selecting an optimal corneal reflection from a plurality of corneal reflection region candidates based on the positional relationship with the pupil.

JP-A-6-274269

  However, the above-described conventional technique has a problem that the corneal reflection region cannot be accurately determined.

  For example, a clear difference does not necessarily occur between the positional relationship between the true corneal reflection region and the pupil and the positional relationship between the spectacle lens reflection region and the pupil. For this reason, as in the prior art, the method of selecting a corneal reflection region from a plurality of corneal reflection region candidates based on the positional relationship with the pupil cannot accurately determine the corneal reflection region.

  The disclosed technology has been made in view of the above, and an object thereof is to provide a corneal reflection determination program, a corneal reflection determination device, and a corneal reflection determination method that can accurately determine a corneal reflection region.

  The corneal reflection determination program disclosed in the present application causes a computer to execute processing for acquiring image data stored in a storage device and extracting an eye region from the image data. Further, the disclosed corneal reflection determination program causes a computer to execute a process of determining a corneal reflection region that appears intermittently among a plurality of detected corneal reflection region candidates.

  According to the corneal reflection determination program disclosed in the present application, there is an effect that the corneal reflection region can be accurately determined.

FIG. 1 is a functional block diagram illustrating the configuration of the corneal reflection determination apparatus according to the present embodiment. FIG. 2 is a diagram for explaining the processing of the tracking unit. FIG. 3 is a diagram illustrating an example of a data structure of the tracking history data storage unit. FIG. 4 is a diagram illustrating an example of a data structure of the open / close data storage unit. FIG. 5 is a diagram for explaining the processing of the determination unit. FIG. 6 is a flowchart (1) showing a processing procedure of the corneal reflection determination apparatus. FIG. 7 is a flowchart (2) showing the processing procedure of the corneal reflection determination apparatus. FIG. 8 is a diagram illustrating an example of a computer that executes a corneal reflection determination program. FIG. 9 is a diagram illustrating an example of a pupil and a corneal reflection region. FIG. 10 is a diagram for explaining the outline of the corneal reflection method. FIG. 11 is a diagram illustrating an example of a cornea reflection region and a spectacle lens reflection region.

  Hereinafter, embodiments of a corneal reflection determination program, a corneal reflection determination apparatus, and a corneal reflection determination method disclosed in the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  First, an outline of the corneal reflection method will be described. FIG. 10 is a diagram for explaining the outline of the corneal reflection method. In FIG. 10, the light source 2 emits infrared rays. 3a represents the cornea, 3b represents the pupil plane, and 3c represents the pupil center. 3d indicates the center of corneal curvature, and 3e indicates the viewing angle. 4 corresponds to an image photographed by the camera, 4a indicates the position of the pupil center of the screen, and 4b indicates the center position of the corneal reflection region of the screen. For example, the pupil center position 4b and the corneal reflection region center position 4b are extracted from the image 4, and the line of sight 5 is detected from the distance between the positions 4a and 4b and the corneal curvature radius.

  The configuration of the corneal reflection determination apparatus according to the present embodiment will be described. FIG. 1 is a functional block diagram illustrating the configuration of the corneal reflection determination apparatus according to the present embodiment. As shown in FIG. 1, the corneal reflection determination apparatus 100 is connected to a camera 50. The corneal reflection determination apparatus 100 includes an image data storage unit 110, an eye region extraction unit 120, a detection unit 130, a tracking unit 140, and a tracking history data storage unit 150. The corneal reflection determination apparatus 100 includes an eye opening / closing determination unit 160, an opening / closing data storage unit 170, a movement identification unit 180, and a determination unit 190.

  The camera 50 is an imaging device that captures images in the imaging range and sequentially stores the captured image data in the image data storage unit 110.

  The image data storage unit 110 is a storage unit that sequentially stores image data photographed by the camera 50. For example, the image data storage unit 110 stores image data for each frame. The image data storage unit 110 corresponds to, for example, a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), or a flash memory, or a storage device such as a hard disk or an optical disk.

  The eye area extraction unit 120 is a processing unit that extracts an eye area from image data. The eye region extraction unit 120 outputs information associating the eye region information with the image data to the detection unit 130 and the eye open / close determination unit 160.

  An example of processing in which the eye region extraction unit 120 extracts an eye region will be described. The eye region extraction unit 120 identifies a circular region from the image data, and when the luminance value around the circular shape is lower than the luminance value of the other region of the image data, the region around the circular shape Are extracted as eye regions. In addition, the eye region extraction unit 120 may extract the eye region using a known technique. For example, the publicly known technology <"Extraction of facial organ features using partial feature templates and global constraints" IEICE Transactions. D-II, Information / System, II-Information Processing J77-D-2 (8), 1601-1609, 1994-08-25>.

  The detection unit 130 is a processing unit that detects a candidate for the corneal reflection region from the eye region on the image data. In the following description, a candidate for the corneal reflection region is referred to as a reflection region candidate. The detection unit 130 outputs information on the reflection area candidate to the tracking unit 140. The information of the reflection area candidate includes the coordinates of the reflection area candidate and the size of the reflection area candidate.

  An example of processing in which the detection unit 130 detects a reflection area candidate will be described. The detection unit 130 scans an eye area on the image data, and detects a partial area having a luminance value equal to or higher than a predetermined value and having a circular shape as a reflection area candidate. For example, the detection unit 130 detects a circular partial region having a luminance value of 250 or more as a reflection region candidate. Then, the detection unit 130 obtains the center of gravity of the reflection area candidate, and uses the obtained center of gravity coordinates as the coordinates of the reflection area candidate.

  The tracking unit 140 is a processing unit that associates the reflection area candidate of the previous frame with the reflection area candidate of the current frame. Then, the tracking unit 140 registers the reflection ID (Identification), the size of the reflection region candidate, and the coordinates in the additional record history data storage unit 150 for each frame. The tracking unit 140 assigns the same reflection ID to each connected reflection area candidate.

  An example of processing in which the tracking unit 140 connects the reflection area candidates will be described. For example, the tracking unit 140 compares the coordinates of the reflection area candidates of the previous and next frames, and connects the reflection area candidates whose difference between the coordinates is less than the threshold.

  FIG. 2 is a diagram for explaining the processing of the tracking unit. FIG. 2 shows an example of reflection area candidates detected in the frames 1 to 3. Frame 1 includes reflection area candidates 1a and 1b. The coordinates (x, y) of the reflection area candidate 1a are set to (72, 38). The coordinates (x, y) of the reflection area candidate 1b are set to (85, 56).

  The frame 2 includes a reflection area candidate 2a. The coordinates (x, y) of the reflection area candidate 2a are set to (82, 57). The frame 3 includes reflection area candidates 3a and 3b. The coordinates (x, y) of the reflection area candidate 3a are set to (72, 38). The coordinates (x, y) of the reflection area candidate 3b are set to (85, 56).

  For example, the threshold value of the difference between x and y is set to 5. In this case, the difference between the coordinates of the reflection area candidate 1b in frame 1 and the reflection area candidate 2a in frame 2 is less than the threshold. Further, the difference between the coordinates of the reflection area candidate 2a of the frame 2 and the reflection area candidate 3b of the frame 3 is less than the threshold value. Therefore, the tracking unit 140 assigns the same reflection ID to the reflection area candidates 1b, 2a, and 3b. The tracking unit 140 assigns different reflection IDs to other reflection area candidates.

  The tracking history data storage unit 150 is a storage unit that stores various types of information regarding the reflection area candidate. The tracking history data storage unit 150 corresponds to, for example, a semiconductor memory device such as a RAM, a ROM, or a flash memory, or a storage device such as a hard disk or an optical disk.

  FIG. 3 is a diagram illustrating an example of a data structure of the tracking history data storage unit. As shown in FIG. 3, the tracking history data storage unit 150 stores the frame number, reflection ID, size, x coordinate, and y coordinate in association with each other. The frame number is a number for identifying a frame. The reflection ID is information for identifying a reflection area candidate. Reflection area candidates corresponding to the same reflection ID are connected reflection candidate areas. The size corresponds to the size of the reflection area candidate.

  The eye open / close determination unit 160 is a processing unit that determines the open / closed state of the eyes. The eye opening / closing determination unit 160 associates the frame number with the eye opening / closing state and registers the frame number in the opening / closing data storage unit 170.

  An example of processing in which the eye open / close determination unit 160 determines the open / closed state of the eyes will be described. The eye open / close determining unit 160 detects the outline of the eye from the eye region, and determines the open / closed state of the eye based on the distance between the upper and lower eyelids. For example, the eye opening / closing determination unit 160 determines that the eyelid is closed when the distance between the upper and lower eyelids is less than a predetermined threshold. On the other hand, the eye opening / closing determination unit 160 determines that the eye is in the open state when the distance between the upper and lower eyelids is equal to or greater than a predetermined threshold.

  The open / close data storage unit 170 is a storage unit that stores the open / closed state of the eye for each frame. The open / close data storage unit 170 corresponds to, for example, a semiconductor memory device such as a RAM, a ROM, or a flash memory, or a storage device such as a hard disk or an optical disk.

  FIG. 4 is a diagram illustrating an example of a data structure of the open / close data storage unit. As shown in FIG. 4, the open / close data storage unit 170 stores the frame number and the open / close state in association with each other. In the example shown in FIG. 4, the frame 1 is in an open state, the frame 2 is in a closed state, and the frame 3 is in an open state.

  The motion identifying unit 180 is a processing unit that identifies the amount of change in facial motion. The motion identifying unit 180 associates the amount of change in face motion with the frame number of the image data from which the amount of change is extracted, and outputs the result to the determination unit 190. The motion specifying unit 180 compares the positions of the feature points of the user's face in the images of the previous and next frames, and specifies the amount of change. The facial feature points correspond to the nose, eyes, mouth, and the like. In addition, when the facial feature point cannot be detected, the motion specifying unit 180 may set the amount of change to a maximum value, for example.

  The determination unit 190 is a processing unit that determines an optimal corneal reflection region from a plurality of reflection region candidates. The determination unit 190 outputs the optimal coordinates of the corneal reflection region to the external device.

  An example of processing in which the determination unit 190 determines an optimal corneal reflection region will be described. First, the determination unit 190 identifies a frame number whose amount of change is less than a predetermined value, and acquires information corresponding to the identified frame number from the tracking history data storage unit 150 and the open / close data storage unit 170.

  Subsequently, the determination unit 190 identifies a reflection ID that appears intermittently in each frame number and appears when the eyes are open. The determination unit 190 determines the corneal reflection region corresponding to the specified reflection ID as the optimum corneal reflection region.

  FIG. 5 is a diagram for explaining the processing of the determination unit. The numbers on the vertical axis correspond to the reflection ID, and the numbers 1 to 3 on the horizontal axis correspond to the frames 1 to 3. In frames 1 and 3, the eye state is an open state, and in frame 2, the eye state is a closed state. In the example shown in FIG. 5, the reflection IDs 1 and 3 appear intermittently at each frame number, and appear when the eyes are open. For this reason, the determination unit 190 outputs the coordinates of the corneal reflection region corresponding to the reflection IDs 1 and 3 to the external device.

  By the way, each said process part 120-140, 160, 180, 190 respond | corresponds to integrated devices, such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array). Moreover, each processing part 120-140, 160, 180, 190 respond | corresponds to electronic circuits, such as CPU and MPU (Micro Processing Unit), for example.

  Next, a processing procedure of the corneal reflection determination apparatus 100 according to the present embodiment will be described. 6 and 7 are flowcharts showing the processing procedure of the corneal reflection determination apparatus. The process illustrated in FIG. 6 is executed when the image data is acquired from the camera 50, for example.

  As shown in FIG. 6, the corneal reflection determination apparatus 100 acquires image data (step S101), and extracts an eye region (step S102). The corneal reflection determination apparatus 100 detects a candidate for a corneal reflection region (step S103), executes a tracking process (step S104), and registers information in the tracking history data storage unit 150 (step S105). The tracking process in step S104 corresponds to the process executed by the tracking unit 140.

  The corneal reflection determining apparatus 100 extracts the outline of the eye (step S106), determines the open / closed state of the eye (step S107), and registers information in the open / close data storage unit 170 (step S108). The corneal reflection determination apparatus 100 proceeds to step S101 again when the process is continued (Yes in step S109). On the other hand, the corneal reflection determination apparatus 100 ends the process when the process is not continued (No in step S109).

  Next, the description proceeds to FIG. The corneal reflection determination apparatus 100 acquires data from the tracking history data storage unit 150 and the open / close data storage unit 170 (step S201), and tracks the reflection ID of each frame (step S202).

  When the reflection ID continuously appears in each frame (step S203, Yes), the corneal reflection determination apparatus 100 excludes the corresponding corneal reflection region (step S204). When there is an unprocessed reflection ID (Yes at Step S205), the corneal reflection determination apparatus 100 proceeds to Step S202 again. On the other hand, when there is no unprocessed reflection ID (No in step S205), the corneal reflection determination apparatus 100 ends the process.

  By the way, when the reflection ID does not appear continuously in each frame (step S203, No), the corneal reflection determination apparatus 100 determines whether or not it appears when the eye state is closed (step S206). .

  The corneal reflection determining apparatus 100 proceeds to step S204 when it appears when the eye is in the closed state (step S206, Yes). On the other hand, if the corneal reflection determining apparatus 100 does not appear when the eye state is closed (step S206, No), the corresponding corneal reflection region is determined as the optimum corneal reflection region (step S207), The process ends.

  Next, the effect of the corneal reflection determination apparatus 100 according to the present embodiment will be described. The corneal reflection determination apparatus 100 detects a corneal reflection region candidate from the eye region of the image data, and determines a corneal reflection region that is intermittently extracted in each frame from the corneal reflection region candidates. For this reason, for example, even when a person is wearing glasses or through a glass window, the cornea reflection region of the person can be accurately determined.

  Further, the corneal reflection determination apparatus 100 determines the open / closed state of the eyelid, and further determines the corneal reflection region using the determination result. Since the cornea reflection region is not detected when the eyelid is in the closed state, the determination accuracy can be improved by specifying the cornea reflection region using this property.

  Further, the corneal reflection determination apparatus 100 determines the corneal reflection region by specifying the amount of change in facial motion and excluding the frame image in which the amount of change is equal to or greater than a predetermined value. When the amount of change in the movement of the face is large, it cannot be expected to accurately follow the reflection ID. For this reason, the determination accuracy can be improved by excluding the image of the frame in which the change amount is less than the predetermined value and determining the corneal reflection region. In addition, since the number of frames of image data to be tracked can be reduced, the processing load can be reduced.

  The determination unit 190 may determine the line-of-sight direction of the imaging target using a plurality of image data after determining the corneal reflection region.

  Next, an example of a computer that executes a corneal reflection determination program that realizes the same function as the corneal reflection determination apparatus 100 illustrated in FIG. 1 will be described. FIG. 8 is a diagram illustrating an example of a computer that executes a corneal reflection determination program.

  As shown in FIG. 8, the computer 200 includes a CPU 201 that executes various arithmetic processes, an input device 202 that receives input of data from a user, and a display 203. The computer 200 includes a reading device 204 that reads a program and the like from a storage medium, and an interface device 205 that exchanges data with other computers via a network. The computer 200 also includes a RAM 206 that temporarily stores various information and a hard disk device 207. The devices 201 to 207 are connected to the bus 208.

  The hard disk device 207 stores an eye area extraction program 207a, a detection program 207b, and a determination program 207c. The CPU 201 reads each program 207 a to 207 c and develops it in the RAM 206. The eye area extraction program 207a functions as an eye area extraction process 206a. The detection program 207b functions as a detection process 206b. The determination program 207c functions as a determination process 206c.

  For example, the eye area extraction process 206 a corresponds to the eye area extraction unit 120. The detection process 206 b corresponds to the detection unit 130 and the tracking unit 140. The determination process 206c corresponds to the determination unit 190.

  Note that the programs 207a to 207c are not necessarily stored in the hard disk device 207 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 200. Then, the computer 200 may read the programs 207a to 207c from these and execute them.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary note 1)
Acquiring a plurality of image data, extracting an eye region from each of the plurality of image data,
Detecting at least one candidate corneal reflection region from each extracted eye region,
Based on the position information where the candidate for the corneal reflection region detected from each of the plurality of image data is detected, correlate the corneal reflection region in each of the plurality of image data,
A corneal reflection determination characterized by causing each process to be executed to select at least a part of corneal reflection region candidates from among corneal reflection region candidates included in each of the plurality of image data based on the associated result program.

(Supplementary note 2) The corneal reflection determination program according to supplementary note 1, wherein a gaze direction of an imaging target is determined using the selected at least a part of the corneal reflection region and the plurality of image data.

(Additional remark 3) The process which selects the candidate of the said at least one part of corneal reflection area | region selects the candidate of corneal reflection area | regions other than the candidate of the corneal reflection area | region which can be matched continuously among several continuous image data. The corneal reflection determination program according to appendix 1 or 2, characterized by:

(Supplementary Note 4) Based on the image data, the computer further executes a process for determining the open / closed state of the eyelid, and the process for determining the corneal reflection region is based on the determination result of the open / closed state of the eyelid. 4. The corneal reflection determination program according to appendix 1, 2, or 3, wherein a corneal reflection region that appears when the eyelid is in an open state is specified among corneal reflection regions detected from a plurality of image data.

(Additional remark 5) The process which specifies the motion of a face further based on the said image data is further performed by a computer, and the process which determines the said corneal reflection area is detected while the variation | change_quantity of a face motion is less than predetermined value The corneal reflection determination program according to any one of Supplementary notes 1 to 4, wherein a candidate for the corneal reflection region that has been selected is selected.

(Supplementary Note 6) An eye region extracting unit that acquires a plurality of image data and extracts an eye region from each of the plurality of image data;
A detection unit for detecting at least one candidate for a corneal reflection region from each extracted eye region;
Based on the position information where the candidate for the corneal reflection region detected from each of the plurality of image data is detected, correlate the corneal reflection region in each of the plurality of image data,
A corneal reflection determination, comprising: a determination unit that selects at least some of the corneal reflection region candidates among the corneal reflection region candidates included in each of the plurality of image data based on the associated results apparatus.

(Additional remark 7) The said determination part further determines the gaze direction of imaging object using the selected said at least one part of cornea reflection area | region and these several image data, The additional remark 6 characterized by the above-mentioned. Corneal reflection determination device.

(Additional remark 8) The said determination part selects the candidate of a corneal reflection area | region other than the candidate of a corneal reflection area | region which can be matched continuously among several continuous image data to additional remark 6 or 7 characterized by the above-mentioned. The corneal reflection determination apparatus described.

(Supplementary Note 9) An eye open / close determining unit that determines the open / close state of the eyelid based on the image data further includes an eye open / close determining unit that determines the open / closed state of the eyelid, and the determination unit is based on a determination result of the eye open / close determining unit. The corneal reflection region according to appendix 6, 7 or 8, wherein the corneal reflection region that appears when the eyelid is in an open state is specified from the corneal reflection regions detected from the plurality of image data. Judgment device.

(Additional remark 10) It has a motion specific part which specifies the motion of a face based on the said image data, The said determination part is the corneal reflection area | region detected while the variation | change_quantity of the facial motion was less than predetermined value The corneal reflection determination apparatus according to any one of Supplementary Notes 6 to 9, wherein the candidate is selected.

(Supplementary note 11) A corneal reflection determination method executed by a computer,
Acquiring a plurality of image data, extracting an eye region from each of the plurality of image data,
Detecting at least one candidate corneal reflection region from each extracted eye region,
Based on the position information where the candidate for the corneal reflection region detected from each of the plurality of image data is detected, correlate the corneal reflection region in each of the plurality of image data,
A corneal reflection determination characterized by executing at least a part of corneal reflection region candidates among the corneal reflection region candidates included in each of the plurality of image data based on the associated results. Method.

(Supplementary note 12) The corneal reflection determination method according to supplementary note 11, wherein the line-of-sight direction of the imaging target is determined using the selected at least part of the corneal reflection region and the plurality of image data.

(Additional remark 13) The process which selects the candidate of the said at least one part corneal reflection area | region selects the candidate of corneal reflection area | regions other than the candidate of the corneal reflection area | region which can be matched continuously among several continuous image data. The method for determining corneal reflection according to Supplementary Note 11 or 12, wherein:

(Additional remark 14) Based on the image data, the computer further executes a process of determining the open / closed state of the eyelid, and the process of determining the corneal reflection region is based on the determination result of the open / closed state of the eyelid. 14. The corneal reflection determination method according to appendix 11, 12, or 13, wherein a corneal reflection region that appears when the eyelid is in an open state is specified from corneal reflection regions detected from a plurality of image data.

(Additional remark 15) The process which specifies the motion of a face further based on the said image data and makes a computer perform the process which determines the said corneal reflection area is detected while the variation | change_quantity of a facial motion is less than predetermined value 15. The method for determining a corneal reflection according to any one of appendices 11 to 14, wherein a candidate for the corneal reflection region that has been selected is selected.

DESCRIPTION OF SYMBOLS 100 Cornea reflection determination apparatus 110 Image data storage part 120 Eye area extraction part 130 Detection part 140 Tracking part 150 Tracking history data storage part 160 Eye opening / closing determination part 170 Opening / closing data storage part 180 Motion specification part 190 Determination part

Claims (5)

On the computer,
Acquiring a plurality of image data, extracting an eye region from each of the plurality of image data,
Detecting at least one candidate corneal reflection region from each extracted eye region,
Based on the position information where the candidate for the corneal reflection region detected from each of the plurality of image data is detected, correlate the corneal reflection region in each of the plurality of image data,
Based on the associated results, select at least some corneal reflection region candidates from the corneal reflection region candidates included in each of the plurality of image data ,
Based on the image data, the open / closed state of the eyelid is determined, and based on the determination result of the open / closed state of the eyelid, out of the corneal reflection regions selected from the plurality of image data, the eyelid is in the open state. A corneal reflection determination program that executes each process for specifying a corneal reflection region that appears in a case .   2. The corneal reflection determination program according to claim 1, wherein the line-of-sight direction of the imaging target is determined using the selected at least a part of the corneal reflection region and the plurality of image data. Based on the image data, the computer further executes a process of identifying a face movement, and the process of selecting the corneal reflection area is performed by detecting the corneal reflection detected while the amount of change in the face movement is less than a predetermined value. The corneal reflection determination program according to claim 1 or 2, wherein a region candidate is selected. An eye area extracting unit that acquires a plurality of image data and extracts an eye area from each of the plurality of image data;
A detection unit for detecting at least one candidate for a corneal reflection region from each extracted eye region;
Based on the position information where the candidate for the corneal reflection region detected from each of the plurality of image data is detected, correlate the corneal reflection region in each of the plurality of image data,
Based on the associated result, at least some of the corneal reflection region candidates are selected from the corneal reflection region candidates included in each of the plurality of image data. Based on the image data , the eyelid is opened or closed. And a determination unit for identifying a corneal reflection region that appears when the fold is in an open state among the corneal reflection regions selected from the plurality of image data based on the determination result of the open / closed state of the fold, A corneal reflection determination apparatus comprising: A corneal reflection determination method executed by a computer,
Acquiring a plurality of image data, extracting an eye region from each of the plurality of image data,
Detecting at least one candidate corneal reflection region from each extracted eye region,
Based on the position information where the candidate for the corneal reflection region detected from each of the plurality of image data is detected, correlate the corneal reflection region in each of the plurality of image data,
Based on the associated results, select at least some corneal reflection region candidates from the corneal reflection region candidates included in each of the plurality of image data ,
Based on the image data, the open / closed state of the eyelid is determined, and based on the determination result of the open / closed state of the eyelid, out of the corneal reflection regions selected from the plurality of image data, the eyelid is in the open state. A method for determining a corneal reflection, comprising: performing each process of identifying a corneal reflection region that appears in a case .

Images