JP2019097807A - Pupil detector - Google Patents

Abstract

To improve robustness of pupil detection.SOLUTION: A pupil detector includes: an image acquisition unit for acquiring a first face image and a second face image of an object person; a template image setting unit for setting a portion of the first face image including the pupil of the object person to a template image; a score calculation unit for calculating a score according to a difference in brightness between a candidate image, which is a portion of the second face image, and the template image for a plurality of the candidate images; an image-for-detection selection unit for selecting an image for detection from a plurality of the candidate images based on the score; and a pupil detection unit for detecting the pupil of the object person based on the template image and the image for detection.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a pupil detection device.

  In recent years, gaze detection technology has been used that detects the pupil of a subject from a bright pupil image and a dark pupil image of the subject's face and estimates the gaze direction of the subject based on the detected coordinates of the pupil. . In this gaze detection technology, in order to detect the pupil more reliably, it is important to correct the positional deviation between the bright pupil image and the dark pupil image which are different in imaging timing. Conventionally, as a method of correcting positional deviation, feature points (nostrils etc.) are respectively detected from the bright pupil image and the dark pupil image, and the positions of the bright pupil image and the dark pupil image are A method of correction has been proposed.

JP 2007-268026 A JP, 2008-29702, A

  However, in the above-described conventional method, when a feature point (such as a nostril) is hidden by a mask or the like, positional deviation of the bright pupil image and the dark pupil image can not be corrected based on the feature point and the pupil may not be detected. was there. That is, the conventional method has a problem that the robustness of pupil detection is low.

  The present invention has been made in view of the above problems, and an object thereof is to improve the robustness of pupil detection.

  The pupil detection device according to one embodiment sets an image acquisition unit for acquiring a first face image and a second face image of a subject, and a part of the first face image including the pupil of the subject as a template image A score calculation unit that calculates, for a plurality of the candidate images, a score according to a luminance difference between the template image setting unit, the candidate image that is a part of the second face image, and the template image; A detection image selection unit that selects a detection image from among a plurality of candidate images based on the above, and a pupil detection unit that detects a pupil of the subject based on the template image and the detection image; Equipped with

  According to each embodiment of the present invention, improvement in robustness of pupil detection can be realized.

The figure which shows an example of the hardware constitutions of a pupil detection apparatus. The figure which shows an example of a function structure of a pupil detection apparatus. The figure which shows an example of a 1st face image, a 2nd face image, and a template image. The flowchart which shows an example of the process which a pupil detection apparatus performs.

  Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. In addition, regarding the description of the specification and drawings according to each embodiment, the same reference numerals are given to components having substantially the same functional configuration, and the overlapping description will be omitted.

  The pupil detection device 100 according to the first embodiment will be described with reference to FIGS. 1 to 4. The pupil detection apparatus 100 according to the present embodiment is an apparatus for detecting the pupil of the subject from the first face image Im1 and the second face image Im2 obtained by photographing the face of the subject. The detection result of the pupil detection device 100 is used by a gaze detection device that detects the gaze of the subject using coordinates of the pupil.

  Hereinafter, a case where the pupil detection device 100 is an on-vehicle device that detects a pupil of a driver of a vehicle will be described as an example. In this case, the driver of the vehicle on which the pupil detection device 100 is mounted corresponds to the subject of the pupil detection device 100.

  First, the hardware configuration of pupil detection apparatus 100 will be described. FIG. 1 is a diagram showing an example of a hardware configuration of pupil detection apparatus 100. As shown in FIG. The pupil detection device 100 of FIG. 1 includes an imaging device 1 and a control device 2.

  The photographing device 1 is a device for photographing a face of a driver and acquiring a face image, and is connected to the control device 2. The photographing device 1 is disposed at a position where the face of the driver can be photographed (in front of the driver's seat). The imaging device 1 of FIG. 1 includes imaging units 11L and 11R, and a plurality of light sources 12L and 12R.

  The photographing unit 11L is a camera for photographing the face of the driver, and is disposed on the left side of the photographing device 1 in FIG. The photographing unit 11R is a camera for photographing the face of the driver, and is disposed on the right side of the photographing device 1 in FIG. Hereinafter, when the imaging units 11L and 11R are not distinguished from one another, they are referred to as the imaging unit 11.

  The light source 12L is a light source that emits light to the driver's face at a predetermined timing when shooting the driver's face, and a plurality of light sources 12L are arranged in the vicinity of the imaging unit 11L. The light source 12R is a light source that emits light to the driver's face at a predetermined timing when shooting the driver's face, and a plurality of light sources 12R are arranged in the vicinity of the photographing unit 11R. The light sources 12L and 12R are, for example, LEDs (Light Emitting Diodes) that emit near infrared light, but are not limited thereto. Hereinafter, when the light sources 12L and 12R are not distinguished from one another, they are referred to as a light source 12.

  In the example of FIG. 1, although the imaging device 1 includes the two imaging units 11, the imaging device 1 may include one or three imaging units 11. In the case where one imaging unit 11 is provided, the imaging apparatus 1 may include the light source 12 substantially coaxial with the imaging unit 11 and the light source 12 non-coaxial. Further, in the example of FIG. 1, the imaging device 1 includes four light sources 12L and 12R, but the number of light sources 12L and 12R can be arbitrarily designed.

  The control device 2 is a device that controls the imaging device 1 and is connected to the imaging device 1. Specifically, the control device 2 controls the photographing by the photographing unit 11 and the light emission of the light source 12. Further, the control device 2 detects the pupil of the driver from the first face image Im1 and the second face image Im2 acquired by the imaging device 1.

  The control device 2 in FIG. 1 includes a central processing unit (CPU) 201, a read only memory (ROM) 202, a random access memory (RAM) 203, a hard disk drive (HDD) 204, an input device 205, and a display. And an apparatus 206. Further, the control device 2 includes a connection interface 207, a communication interface 208, and a bus 209.

  The CPU 201 executes each program to control each hardware configuration of the control device 2 to realize the function of the pupil detection device 100.

  The ROM 202 stores programs executed by the CPU 201 and various data.

  The RAM 203 provides the CPU 201 with a work area.

  The HDD 204 stores programs executed by the CPU 201 and various data.

  The input device 205 inputs information to the control device 2 according to the operation of the user including the driver. The input device 205 is, for example, a touch panel, a keyboard, a mouse, a hardware button, or the like.

  The display device 206 displays a screen according to the operation of the user including the driver. The display device 206 is, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, or the like.

  The connection interface 207 is an interface for connecting the control device 2 to the imaging device 1. The control device 2 controls the imaging device 1 via the connection interface 207 and acquires an image (image data) from the imaging device 1.

  The communication interface 208 is an interface for connecting the control device 2 to a network such as a controller area network (CAN), the Internet, or a local area network (LAN). The control device 2 communicates with an external device (such as an ECU (Electronic Control Unit)) on the network via the communication interface 208.

  The bus 209 mutually connects the CPU 201, the ROM 202, the RAM 203, the HDD 204, the input device 205, the display device 206, the connection interface 207, and the communication interface 208.

  In addition, in the example of FIG. 1, the case where the control apparatus 2 is comprised by a vehicle-mounted computer is assumed. However, the control device 2 may be configured in one chip by an IC such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specified Integrated Circuit). In this case, the control device 2 does not include the HDD 204, the input device 205, and the display device 206. Further, the control device 2 may be configured by a computer such as a PC (Personal Computer), a server, a tablet terminal, a smartphone, etc. connected via a network to the imaging device 1 mounted on a vehicle.

  Next, the functional configuration of pupil detection device 100 will be described. FIG. 2 is a diagram showing an example of a functional configuration of the pupil detection device 100. As shown in FIG. The pupil detection device 100 of FIG. 2 includes an image acquisition unit 21, an image storage unit 22, a template image setting unit 23, a score calculation unit 24, a detection image selection unit 25, and a pupil detection unit 26. . Each functional configuration is realized by the CPU 201 of the control device 2 executing a program and cooperating with each hardware configuration of the pupil detection device 100.

  The image acquisition unit 21 controls the imaging device 1 to acquire a first face image Im1 and a second face image Im2 obtained by imaging the driver's face at different imaging timings. Specifically, the image acquisition unit 21 causes the imaging device 1 to capture the face of the driver at a predetermined imaging interval, and acquires the first face image Im1 and the second face image Im2 at the imaging interval. The imaging interval is, for example, 1 msec or more and 100 msec or less. The first face image Im1 is one of a bright pupil image and a dark pupil image, and the second face image Im2 is the other of the bright pupil image and the dark pupil image.

  The bright pupil image is a face image in which the driver's pupil is photographed brightly. The bright pupil image is obtained by capturing an image of the driver's face with the imaging unit 11 substantially coaxial with the light source 12 in a state where the light is emitted from the light source 12 to the driver's face. By photographing in this manner, the light from the light source 12 is reflected in the direction of the photographing unit 11 by the retina of the driver, so that the pupil of the driver is photographed brightly. In the imaging device 1 of FIG. 1, a bright pupil image is obtained by imaging the driver's face with the imaging unit 11L in a state where light is emitted from the light source 12L to the driver's face. In addition, a light pupil image is acquired by photographing the driver's face with the imaging unit 11R in a state where light is emitted from the light source 12R to the driver's face.

  The dark pupil image is a face image in which the driver's pupil is photographed darker than the bright pupil image. The dark pupil image is obtained by photographing the driver's face with the imaging unit 11 that is not coaxial with the light source 12 in a state in which light is emitted from the light source 12 to the driver's face. By photographing in this manner, the light from the light source 12 is reflected by the driver's retina in a direction different from that of the photographing unit 11, so that the pupil of the driver is photographed dark. In the imaging device 1 of FIG. 1, a dark pupil image is obtained by imaging the driver's face with the imaging unit 11R in a state where light is emitted from the light source 12L to the driver's face. In addition, a dark pupil image is obtained by photographing the driver's face with the imaging unit 11L in a state where the light is emitted from the light source 12R to the driver's face.

  The image storage unit 22 stores images such as a first face image Im1, a second face image Im2, a template image Imt, and a difference image.

  The template image setting unit 23 sets a part of the first face image Im1 stored in the image storage unit 22 as a template image Imt. The template image Imt is an image used for template matching described later. The template image setting unit 23 sets, as the template image Imt, a portion of the first face image Im1 that has an arbitrary size including the pupil of the driver. As a method of setting a template image Imt including a pupil, a face is detected from the first face image Im1 by machine learning or detection of the contour of a face, and a portion where the pupil is considered to be included in the face (for example, It is conceivable to set half) as the template image Imt. The template image Imt is preferably smaller in size. This can speed up template matching.

  The score calculation unit 24 calculates a score according to the luminance difference between the partial images of the plurality of second face images Im2 (hereinafter referred to as “candidate image Imc”) and the template image Imt by template matching. The candidate image Imc is an image to be a candidate for the detection image Imd described later. The score calculation unit 24 selects a partial image of the same size as the template image Imt as the candidate image Imc from the second face image Im2, and generates a difference image of the selected candidate image Imc and the template image Imt. The average value or the integrated value of the luminance of the generated difference image is calculated as the score of the candidate image Imc. As the candidate image Imc and the template image Imt are more similar, the brightness of the difference image is smaller, so the score is smaller. That is, the score is smaller as the candidate image Imc and the template image Imt are more similar.

  The difference image is an image obtained by subtracting the dark pupil image from the bright pupil image. When the bright pupil image is the first face image Im1 and the dark pupil image is the second face image Im2, the difference image is an image obtained by subtracting the partial image of the second face image Im2 from the template image Imt. Conversely, when the dark pupil image is the first face image Im1 and the bright pupil image is the second face image Im2, the difference image sets the template image Imt from the first face image Im1, which is the dark pupil image, The partial image of the second face image Im2 is subtracted from the template image Imt. The score calculation unit 24 stores the generated difference image in the image storage unit 22 in association with identification information of the candidate image Imc corresponding to the difference image. The identification information of the candidate image Imc includes position information (coordinates) identifying the candidate image Imc.

  The score calculation unit 24 may store the difference image corresponding to all the candidate images Imc in the image storage unit 22. In addition, the score calculation unit 24 may store only the difference image corresponding to the candidate image Imc having the smallest score in the image storage unit 22, or corresponds to the candidate image Imc having the smallest score and the template image Imt. Only the difference image corresponding to the candidate image Imc may be stored in the image storage unit 22. The candidate image Imc corresponding to the template image Imt is a candidate image Imc having the same position as the template image Imt. As described above, the storage capacity required for the image storage unit 22 can be reduced by storing only the difference image corresponding to a part of the candidate images Imc.

  The score calculation unit 24 selects the candidate image Imc while shifting each predetermined pixel (one pixel or a plurality of pixels) in the matching range R based on the position of the template image Imt in the first face image Im1, and selects each candidate image Calculate Imc's score. The score calculation unit 24 associates the identification information of the candidate image Imc with the score of the candidate image Imc, and notifies the detection image selection unit 25 of the correspondence.

  FIG. 3 is a diagram showing an example of the first face image Im1, the second face image Im2, and the template image Imt. In the example of FIG. 3, the rectangular range from (n1, m1) to (n2, m2) in the first face image Im1 is set as the template image Imt, and (n1-p, m1-q) in the second face image Im2 A rectangular range from to (n 2 + p, m 2 + q) is set as the matching range R. Thus, the matching range R is set based on the position of the template image Imt in the first face image Im1.

  The score calculation unit 24 calculates a score while shifting the template image Imt by a predetermined number of pixels in the matching range R. In other words, the score calculation unit 24 selects the candidate image Imc in the matching range R while shifting each predetermined pixel, and calculates the score of each of the selected candidate images Imc. In the example of FIG. 3, when p and q are 2 pixels and the predetermined pixel is 1 pixel, the score calculation unit 24 calculates the scores of 25 candidate images Imc.

  As described above, since the first face image Im1 and the second face image Im2 differ in shooting timing by the shooting interval, the driver's face included in the first face image Im1 and the second face image Im2 is the example of FIG. The position slightly shifts depending on the shooting interval. The pupil detection apparatus 100 searches for a candidate image Imc most similar (smallest score) to the template image Imt in order to correct the positional deviation of the face included in the first face image Im1 and the second face image Im2. It is considered that the positional deviation between the template image Imt and the candidate image Imc most similar to the template image Imt corresponds to the positional deviation of the face included in the first face image Im1 and the second face image Im2. In order to For this reason, it is preferable that the matching range R (a search range of the candidate image Imc most similar to the template image Imt) is set wider than the template image Imt by at least the displacement of the face. On the other hand, the longer the matching range R, the longer the time required for template matching. Therefore, in order to accelerate template matching, it is preferable that the matching range R be narrow.

  As a result, it is preferable that the matching range R is set wider than the template image Imt by the displacement of the face and as narrow as possible. Preferably, the matching range R is set wider as the imaging interval is longer and smaller as the imaging interval is shorter so as to enable setting of the matching range R as described above. The score calculation unit 24 may be capable of changing the matching range R in accordance with the imaging interval.

  The detection image selection unit 25 selects the detection image Imd from the candidate images Imc for which the score calculation unit 24 has calculated the score, based on the score calculated by the score calculation unit 24. The detection image Imd is an image for detecting a driver's pupil. The detection image selection unit 25 notifies the pupil detection unit 26 of identification information of the candidate image Imc selected as the detection image Imd.

  As a first selection method of selecting the detection image Imd, a method of selecting the candidate image Imc having the smallest score as the detection image Imd can be considered. According to the first selection method, the candidate image Imc most similar to the template image Imt can be easily selected.

  As a second selection method of selecting the detection image Imd, a method of selecting a candidate image Imc having the smallest score and less than a threshold as the detection image Imd can be considered. According to the second selection method, when the smallest score is equal to or higher than the threshold, the candidate image Imc corresponding to the template image Imt is selected as the detection image Imd.

  According to the second selection method, it is possible to improve the pupil detection accuracy more than the first selection method. The reason is as follows.

  When the positional deviation of the face included in the first face image Im1 and the first face image Im2 is large due to the vibration of the vehicle or the driver's action, each candidate image Imc is compared with the dissimilar template image Imt. Score increases overall. However, in the first selection method, even in such a case, the candidate image Imc with the smallest score is selected as the detection image Imd. If a pupil is detected based on the template image Imt and the dissimilar candidate image Imc, there is a possibility that the detection accuracy of the pupil may be lower than in the case where the pupil is detected without using the candidate image Imc. According to the second selection method, even if the candidate image Imc having the smallest score is not selected as the detection image Imd if the score is equal to or higher than the threshold, the non-similar candidate image Imc is caused It is possible to suppress a drop in pupil detection accuracy.

  Even when the positional deviation of the face included in the first face image Im1 and the first face image Im2 is small, the score of the candidate image Imc most similar to the template image Imt is larger than the threshold due to the difference in the luminance of the pupil. It is conceivable that In this case, in the second selection method, the candidate image Imc most similar to the template image Imt is not selected as the detection image Imd, and the candidate image Imc corresponding to the template image Imt is selected. It may decrease. However, in this case, since the pupil can be detected with high accuracy based on the candidate image Imc corresponding to the template image Imt because the positional deviation is small originally, it is considered that the decrease in the pupil detection accuracy is sufficiently small. .

  As a result, according to the second selection method, compared to the first selection method, a decrease in pupil detection accuracy due to the candidate image Imc that is dissimilar to the template image Imt is suppressed, so that the pupil detection accuracy Can be improved.

  When the second selection method is adopted, it is preferable that the detection image selection unit 25 make the threshold smaller as the imaging environment is brighter, and makes the threshold larger as the imaging environment is darker. This is because it is considered that when the imaging environment is bright, the driver's pupil contracts and the score of the candidate image Imc most similar to the template image Imt decreases. The detection image selection unit 25 can use an average value, an integrated value, or the like of the luminance of the first face image Im1 and the second face image Im2 as the brightness of the imaging environment.

  Note that the method of selecting the detection image Imd is not limited to the first selection method and the second selection method.

  The pupil detection unit 26 detects the pupil of the driver based on the template image Imt and the detection image Imd. Specifically, the pupil detection unit 26 generates a difference image between the template image Imt and the detection image Imd, generates a binarized image obtained by binarizing the difference image, and generates a binarized image based on the binarized image. To detect the coordinates of the driver's pupil.

  The pupil detection unit 26 acquires the detection image Imd from the second face image Im2 based on the identification information notified from the detection image selection unit 25 and acquires the template image Imt from the image storage unit 22. A binarized image can be generated based on the template image Imt and the detection image Imd.

  In addition, the pupil detection unit 26 acquires a difference image corresponding to the candidate image Imc selected as the detection image Imd from the image storage unit 22 based on the identification information notified from the detection image selection unit 25. It is also good. The difference image corresponds to a difference image generated based on the template image Imt and the detection image Imd. The pupil detection unit 26 can speed up the pupil detection processing by generating a binarized image based on the difference image acquired from the image storage unit 22.

  When the detection image selection unit 25 adopts the first selection method, only the difference image corresponding to the candidate image Imc having the minimum score may be stored in the image storage unit 22. In addition, when the detection image selection unit 25 adopts the second selection method, only the difference image corresponding to the candidate image Imc having the minimum score and the candidate image Imc corresponding to the template image Imt is stored in the image storage unit 22. You can save it to Thereby, the storage capacity required for the image storage unit 22 can be reduced while speeding up the pupil detection processing.

  As described above, since the candidate image Imc most similar to the template image Imt is selected as the detection image Imd by the detection image selection unit 25, the position of the pupil in the template image Imt and the pupil in the detection image Imd The position substantially matches. By generating a binarized image based on such a template image Imt and the detection image Imd, the pupil detection unit 26 can generate a binarized image in which the pupil of the driver is drawn white. The pupil detection unit 26 may detect the center of gravity of the pupil drawn white and the like as the coordinates of the pupil. The coordinates of the pupil detected by the pupil detection unit 26 can be used by a visual axis detection device or the like in the subsequent stage.

  Next, the process which the pupil detection apparatus 100 performs is demonstrated. FIG. 4 is a flowchart showing an example of processing executed by the pupil detection device 100. The pupil detection device 100 executes the process of FIG. 4 at predetermined time intervals. Hereinafter, the case where the detection image selection unit 25 adopts the second selection method will be described as an example.

  When the execution timing of the processing comes, the image acquisition unit 21 causes the imaging device 1 to capture the bright pupil image and the dark pupil image of the driver at predetermined imaging intervals, and acquires the first face image Im1 and the second face image Im2. (Step S101). The image acquisition unit 21 stores the acquired first face image Im1 and second face image Im2 in the image storage unit 22.

  When the first face image Im1 and the second face image Im2 are stored in the image storage unit 22, the template image setting unit 23 converts part of the first face image Im1 stored in the image storage unit 22 into the template image Imt. It sets (step S102). The setting method of the template image Imt is as described above. The template image setting unit 23 stores the set template image Imt in the image storage unit 22.

  When the template image Imt is stored in the image storage unit 22, the score calculation unit 24 selects a part of the second face image Im2 stored in the image storage unit 22 as a candidate image Imc (step S103). The method of selecting the candidate image Imc is as described above.

  Next, the score calculation unit 24 generates a difference image between the selected candidate image Imc and the template image Imt stored in the image storage unit 22, and calculates the score of the candidate image Imc based on the difference image. (Step S104). The method of calculating the score is as described above. The score calculation unit 24 notifies the detection image selection unit 25 of the calculated score and the identification information of the candidate image Imc for which the score is calculated.

  Subsequently, the score calculation unit 24 checks whether the candidate image Imc corresponds to the template image Imt, that is, the positions of the template image Imt and the candidate image Imc are the same (step S105). When the candidate image Imc corresponds to the template image Imt (YES in step S105), the score calculation unit 24 sets the difference image generated from the candidate image Imc as a difference image corresponding to the candidate image Imc corresponding to the template image Imt. The image information is stored in the image storage unit 22 in association with the identification information of the candidate image (step S106).

  Similarly, the score calculation unit 24 confirms whether the score of the candidate image Imc is the minimum among the scores calculated so far (step S107). When the score of the candidate image Imc is the smallest (YES in step S107), the score calculation unit 24 sets the difference image generated from the candidate image Imc as the difference image corresponding to the candidate image Imc with the smallest score, the candidate image In association with the identification information, and stored in the image storage unit 22 (step S108). When the difference image corresponding to the candidate image Imc having the smallest score is already stored, the score calculation unit 24 may overwrite and store the new difference image on the stored difference image. Steps S105 and S107 are in random order.

  Thereafter, the score calculation unit 24 confirms whether all the candidate images Imc selectable in the matching range R have been selected (step S109). If there is an unselected candidate image Imc (NO in step S109), the process returns to step S103. On the other hand, if all candidate images Imc have been selected (YES in step S109), the process proceeds to step S110. Thereby, the scores of all the candidate images Imc selectable in the matching range R are calculated.

  When the scores of all the candidate images Imc selectable in the matching range R are calculated, the detection image selection unit 25 determines that the minimum score among the plurality of scores notified from the score calculation unit 24 is less than the threshold value. It is determined whether or not (step S110). If the minimum score is less than the threshold (YES in step S110), the detection image selecting unit 25 selects the candidate image Imc having the minimum score as the detection image Imd (step S111). On the other hand, when the minimum score is equal to or higher than the threshold (NO in step S110), the detection image selecting unit 25 selects the candidate image Imc corresponding to the template image Imt as the detection image Imd (step S112). The detection image selection unit 25 notifies the pupil detection unit 26 of identification information of the selected detection image Imd.

  When notified of the identification information of the detection image Imd, the pupil detection unit 26 acquires a difference image corresponding to the candidate image Imc selected as the detection image Imd from the image storage unit 22 based on the identification information. (Step S113).

  Thereafter, the pupil detection unit 26 generates a binarized image based on the acquired difference image, and detects the coordinates of the pupil of the driver based on the binarized image (step S114).

  As described above, the pupil detection apparatus 100 selects the detection image Imd most similar to the template image Imt without using the feature points (nostrils etc.) of the subject, and selects the template image Imt and the detection image Imd. The pupil of the subject can be detected based on that. Selecting the detection image Imd most similar to the template image Imt corresponds to correcting the positional deviation of the first face image Im1 and the second face image Im2. Therefore, even if the feature point of the subject is hidden by a mask or the like, the pupil detection device 100 corrects the positional deviation of the first face image Im1 and the second face image Im2 and detects the pupil of the subject. can do. In other words, according to the present embodiment, the robustness of pupil detection based on the bright pupil image and the dark pupil image can be improved.

  Further, according to the present embodiment, the size of the template image Imt can be set small because it suffices to include the pupil of the subject. Further, the matching range R can be set narrow according to the imaging interval. Thus, template matching can be speeded up by reducing the template image Imt and narrowing the matching range R.

  In addition, since the difference image corresponding to the candidate image Imc for which the score is calculated is stored in the image storage unit 22 at the time of calculation of the score, the pupil detection unit 26 uses the difference image stored in the image storage unit 22. And the pupil can be detected. In other words, the pupil detection unit 26 does not have to generate a difference image again after the selection of the detection image Imd. Therefore, according to the present embodiment, the pupil detection process can be speeded up.

  Note that the present invention is not limited to the configurations shown here, such as combinations with other elements in the configurations and the like described in the above embodiments. These points can be changed without departing from the spirit of the present invention, and can be appropriately determined according to the application form.

1: imaging device 2: control device 11: imaging unit 12: light source 21: image acquisition unit 22: image storage unit 23: template image setting unit 24: score calculation unit 25: detection image selection unit 26: pupil detection unit 100: Pupil detection device Im1: first face image Im2: second face image Imt: template image Imc: candidate image Imd: detection image R: matching range

Claims (7)

An image acquisition unit for acquiring a first face image and a second face image of a subject,
A template image setting unit configured to set a part of the first face image including the pupil of the subject as a template image;
A score calculation unit configured to calculate, for a plurality of candidate images, a score according to a luminance difference between a candidate image that is a part of the second face image and the template image;
A detection image selection unit that selects a detection image from among the plurality of candidate images based on the score;
A pupil detection unit that detects the pupil of the subject based on the template image and the detection image;
A pupil detection device comprising: The pupil detection device according to claim 1, wherein the detection image selection unit selects the candidate image with the smallest score as the detection image. The pupil detection apparatus according to claim 1, wherein the detection image selection unit selects the candidate image having the smallest score and the score less than a threshold as the detection image. The pupil detection apparatus according to claim 1, wherein the detection image selection unit selects the candidate image corresponding to the template image as the detection image when the smallest score is equal to or more than the threshold. The pupil detection device according to claim 3 or 4, wherein the detection image selection unit reduces the threshold as the luminances of the first face image and the second face image increase. The pupil detection unit according to any one of claims 1 to 5, wherein the pupil detection unit detects the pupil based on a binarized image obtained by binarizing a difference image between the template image and the detection image. A pupil detection device as described. The first face image is one of a bright pupil image and a dark pupil image, and the second face image is the other of the bright pupil image and the dark pupil image. The pupil detection device according to item 1.

Images