JP4452832B2 - Method and apparatus for detecting pupils - Google Patents

Description

  The present invention relates to a method for detecting a pupil of a subject based on a photographed image and an apparatus therefor.

  The technology for detecting the subject's pupil is, for example, preventing a drowsy driving by detecting the subject's line of sight and changes in the line of sight while driving a car, or using a finger by the movement of the subject's pupil It can also be used for a pupil mouse that performs input to a computer. The inventor has developed a technique for detecting a pupil from an image photographed by a camera. For example, Patent Document 1 discloses a technique for detecting a subject's line of sight from a position between a pupil and a corneal reflection point. Patent Document 2 discloses a technique in which the position coordinate of the detected pupil is used as a cursor position control signal. Patent Document 3 discloses a technique for determining the direction of the head by detecting the positions of the pupil and nostril. Patent Document 4 discloses a technique for determining the three-dimensional position of the viewpoint viewed by the subject by detecting the positions of the pupil and the corneal reflection point.

  In these techniques, for detection of the pupil, a method is used in which the pupil portion is distinguished from the surrounding image by subtracting the bright pupil image and the dark pupil image captured by the camera. When a light source such as near-infrared light is provided near the camera's aperture and the subject's face is irradiated with light along the optical axis of the camera, the light reaches the retina from the pupil and is reflected, and the lens and cornea are reflected. Go back to the camera opening. In this image, the pupil is photographed brightly, and the image is called a bright pupil image. On the other hand, if the subject's face is illuminated with light from a light source away from the camera opening, the light reflected from the retina hardly enters the camera opening, so the pupil is photographed darkly, and the image is darkened. This is called a pupil image. The size of the pupil changes depending on the brightness of the surroundings, and becomes small and difficult to detect particularly in bright places. In addition, since a part of the glasses near the pupil causes reflection in a spectacle wearer, it is difficult to detect the pupil from a single image using either the bright pupil or the dark pupil. However, if the difference between the image obtained by photographing the bright pupil and the image obtained by photographing the dark pupil is obtained, the surrounding portions other than the pupil portion have almost the same brightness in both images. Only the pupil with a difference will be highlighted. Thereby, the pupil can be easily detected, and the cost of pupil detection can be reduced and the robustness can be improved.

  Depending on the surrounding brightness and other conditions, the image of the pupil part is not necessarily brighter than the surrounding image even in the case of the bright pupil. It is not always darker. The bright pupil and the dark pupil in this specification have a relative brightness difference between the pupils in the two images taken at that time, and the bright pupil is relatively brighter than the dark pupil. That's what it means.

Furthermore, the present inventor shot the face of the subject in a state in which the light sources having different wavelengths of retina are simultaneously turned on without switching on and turning on the two light sources. Patent applications 1 and 2 relating to a technique for simultaneously obtaining a bright pupil image and a dark pupil image with different wavelengths and performing a difference between the images.
[Patent Document 1] JP-A-2005-185431
[Patent Document 2] JP 2005-182247 A
[Patent Document 3] Japanese Patent Application Laid-Open No. 2005-266868
[Patent Document 4] Japanese Patent Application Laid-Open No. 2005-198743
[Patent Application 1] Japanese Patent Application No. 2005-241728
[Patent Application 2] Japanese Patent Application No. 2005-306961

However, in the techniques of Patent Documents 1 to 4, since the light source for bright pupil photography and the two light sources for dark pupil photography are alternately turned on and photographed, the photographing of the bright pupil image and the dark pupil image involves a time difference. If the subject's face moves during that time, the positions of the bright and dark pupils in both images will shift, and the position of the pupil may not be detected accurately due to the image difference. For example, when an odd field and an even field constituting one frame every 1/30 seconds in NTSC camera shooting are a bright pupil image and a dark pupil image, respectively, 1/60 seconds are required for shooting both images. If there is a time difference and the position of the pupil in both images shifts to a state where they do not overlap due to the movement of the face between them, there is no effect of differentiating the images. As shown in FIG. 12A, when the bright pupil image P1 and the dark pupil image P2 are partially shifted, the difference effect appears only in the overlapping hatched portion. If this further deviates, there will be no overlapping portion and no difference effect will be produced, making pupil detection difficult. In addition, if the face moves while the shutter for photographing bright pupils and dark pupils is opened, the pupil image is blurred as shown in FIG. Even in the bright pupil image P1 and the dark pupil image P2 blurred as shown in (C), if a part of them overlaps, a temporary difference effect appears in the hatched portion. However, as shown in FIG. 13, if there is a high-luminance part such as the reflection of glasses near the pupil, the images d1 and d2 are erased by taking the difference if the two images match. However, if the two images are shifted, even if the difference is made, it remains as a high-luminance portion like a shaded portion, and thus it may not be distinguished from the pupil images P1 and P2. Thus, if the bright pupil image and the dark pupil image are shifted, it becomes difficult to detect the pupil with high accuracy.
In addition, according to the techniques of Patent Applications 1 and 2, light of different wavelengths is simultaneously irradiated, and a bright pupil image and a dark pupil image are obtained at the same wavelength, so that there is no time difference, but wavelength separation is performed. Problems such as the need for equipment and two cameras for photographing at two wavelengths, and the need for work for wavelength separation arise.

  The present invention has been made to solve these problems, and the position of the pupil part due to the time difference between the image obtained by photographing the bright pupil and the image obtained by photographing the dark pupil without requiring a particularly complicated apparatus or work. It is an object of the present invention to provide a highly robust pupil detection method and apparatus that eliminates the deviation.

  In order to achieve such an object, the method for detecting a pupil of the present invention is a method for detecting a pupil by subtracting a bright pupil image from a dark pupil image, and photographing a bright pupil at a certain point in time. The amount of deviation of the nostril position between the captured image and the image of the dark pupil taken at or after that point is detected, and the image of the bright pupil or the image of the dark pupil is Position correction is performed so as to cancel the shift by an amount corresponding to the shift amount, and both images after the position correction are differentiated.

  According to the present invention, the deviation amount of the nostril position in two images obtained by photographing the bright pupil and the dark pupil is detected, and the image obtained by photographing the bright pupil or the image obtained by photographing the dark pupil is converted into the deviation amount of the nostril position. By performing position correction that moves in a direction that cancels the shift by the amount corresponding to, the pupil portions of both images can be substantially matched, and then the difference between the images is made. Therefore, even though there is a time difference between the acquisition of both images, the bright pupil and the dark pupil are differentiated in a state where the positions coincide with each other and are embossed from the surroundings, so that the pupil detection is performed with high accuracy. The reason for using the nostril as a standard for detecting the deviation is that in the case of a nostril, unlike a pupil that is closed by the eyelids, is reduced by brightness, or has glasses, In each of the images taken of the dark pupil, it is possible to detect stably as a dark part with respect to the surroundings, and when moving the face, it moves in the same parallel as the pupil, so use the nostril This is because it is not necessary to attach a marker or the like for detection reference to the face. Here, the amount corresponding to the shift amount of the nostril position, which is the amount of position correction, may be the same amount as the shift amount of the nostril position. If there is a correspondence relationship other than 1, the amount may be taken into consideration. In addition, the image that is moved for position correction and subsequently subjected to the difference need not be the entire image obtained by photographing the bright pupil or the entire image obtained by photographing the dark pupil.

  Further, in the present invention, when the image of the bright pupil moving in the position correction or the image of the dark pupil is an image of the window area set around the pupil, it corresponds to the displacement of the nostril position. Since the image area for which the position correction is performed and the analysis for detecting the pupil is performed can be the minimum necessary image area around the pupil, the calculation efficiency of the pupil detection can be improved.

  Further, in the present invention, when the position correction is to move the previously captured image to the position of the later captured image, the pupil in the later captured image is obtained. That is, the position of the pupil in the latest image can be obtained, and the subsequent pupil can be detected using this as a reference.

  Further, in the present invention, prior to detection of the displacement amount of the nostril position, the pupil position is detected by subtracting the bright pupil image from the dark pupil image, and the nostril circumference is determined based on the detected pupil position. If the window is set and the process of detecting the nostril by searching in the window is involved, initial nostril detection becomes easy. In other words, once the nostril can be detected, it can be stably detected without using the difference in the image, but under certain conditions such as the face position being almost stable, the difference in the image is used to make the pupil. In that case, based on the detected pupil position, set a window where the nostril position is expected and search for the nostril within that area. Is efficient.

  The pupil detection device of the present invention is a device for detecting a pupil by subtracting a bright pupil image from a dark pupil image, and means for photographing a bright pupil and a dark pupil, and a bright light at a certain time. Means for detecting the amount of deviation of the position of the nostril between the image obtained by photographing the pupil and the image obtained by photographing the dark pupil at the subsequent time or the time before the image; and the image obtained by photographing the bright pupil or the image obtained by photographing the dark pupil. Means for correcting the position of movement in a direction corresponding to the amount of deviation of the nostril position, and means for determining the position of the pupil by subtracting the two images after position correction.

  According to the present invention, the detection unit detects a shift amount of the nostril position in the two images obtained by photographing the bright pupil and the dark pupil by the photographing unit, and an image obtained by photographing the bright pupil or an image obtained by photographing the dark pupil, By performing position correction that moves in a direction that cancels the shift by an amount corresponding to the shift amount of the nostril position, the pupil portions of both images can be substantially matched, and then the image By calculating the difference, the position of the pupil can be obtained. Therefore, although there is a time difference in the acquisition of both images, the bright pupil and the dark pupil are differentiated in a state in which the positions coincide with each other and are embossed from the surroundings, so that the pupil position is accurately detected.

  According to the present invention, a highly robust pupil can be achieved by eliminating a shift in the position of the pupil due to a time difference between an image obtained by photographing a bright pupil and an image obtained by photographing a dark pupil without requiring a particularly complicated device or work. Detection method and apparatus can be provided.

  Hereinafter, preferred embodiments of a method and apparatus for detecting a pupil according to the present invention will be described in detail with reference to the drawings. First, the basic technique of this embodiment will be described with reference to FIG. FIG. 1A is a conceptual diagram showing an image obtained by photographing a bright pupil and a dark pupil of a subject. Two near-infrared light sources that illuminate the face of the subject are two light sources arranged near and far from the camera opening. When an NTSC system camera is used as a photographing means, in the NTSC system, one frame obtained per 30 frames is composed of an odd field with only odd lines and an even field with only even lines. And even field images are alternately obtained at 1/60 second intervals. Therefore, if the light source located near the camera aperture is lit when shooting an odd field, and the light source located far from the camera aperture is lit when shooting an even field, the bright pupil is shot in the odd field, In the even field, the dark pupil is photographed.

  In FIG. 1A, the face 1a of the subject is photographed in the odd field image thus photographed, in which the left and right pupils 21a and 22a, which are bright pupils, and the left and right nostrils 31a are captured. , 32a exist. Here, the nostrils 31a and 32a can be detected by binarizing the image brightness by a discriminant analysis method or the like from one image if the position is predicted to some extent, but the pupils 21a and 22a are detected. As described above, it is difficult to detect only from this one image. Therefore, the difference between the left and right pupils 21b and 22b, which are dark pupils in the image of the even field at the next time point, is detected and the pupil is detected, but the face is moved to the position of 1b in 1/60 seconds. The bright pupils 21a and 22a and the dark pupils 21b and 22b are displaced as shown in the figure, and even if the difference is taken, the pupil is not relieved with respect to the surroundings. Cannot be detected.

  However, since the positions of the nostrils 31b and 31b can be detected even in the images of the dark pupils 21b and 22b, the positions of the nostrils are shifted from 31a and 32a to 31b and 32b as indicated by arrows between the two images. The amount can be determined. Then, after correcting the image obtained by photographing the bright pupils 21a and 22a to the position of the image obtained by photographing the dark pupils 22a and 22b by the amount of the deviation, the position of the pupil is determined by taking the image difference. Since they match, it is possible to detect whether the pupil is embossed on the difference image obtained at the image position of the even field. In that case, if the entire image is moved, the burden of calculation is large, and if the image to be analyzed is wide, a portion that is confused with the pupil may be an analysis target. Therefore, as shown in FIG. If only the images in the windows 41a and 42a that are predicted to include the bright pupils 21a and 22a are corrected by moving the nostril displacement amounts as 41b and 42b, and the difference between the regions is calculated. Good.

  In this case, even if the average value of the right and left nostril deviations is used as the correction value for the nostril deviation, the right nostril deviation amount is used to correct the right pupil position and the left nostril deviation amount. May be used to correct the position of the left pupil, or may be used to correct the position of both pupils by detecting the amount of deviation of one nostril. In addition, when it is predicted that the movement amounts of the nostril and the pupil are almost equal, the direction of the nostril deviation and the deviation value may be used as they are for correcting the pupil position. In order to easily detect the position of the nostril, it is preferable to take a picture of the nostril from the lower side with the position of the camera below. Since there is a tendency that the value of the direction and the deviation does not completely coincide with the deviation of the pupil position, the position correction may be performed in consideration of such a tendency. Further, the bright pupil or the dark pupil may be imaged in either the odd field or the even field. In order not to blur the image, it is preferable that the time during which the shutter of the camera is opened is short, that is, the shutter speed is high.

  In FIG. 1, the image correction of the position correction and the image difference between the images of the bright pupils 21 a and 22 a and the image of the dark pupils 21 b and 22 b at the next time point are described as one unit. In the subsequent steps, the same processing may be performed for an image obtained by photographing the bright pupil and an image obtained by photographing the dark pupil as one unit. In this case, since the pupil in the image of the dark pupil, for example, the image of the dark pupil after one unit is obtained, if the dark pupil image can be obtained in 1/30 seconds, the detection of the pupil is also performed at that interval. .

  Alternatively, as shown in FIG. 2, the position correction and the image difference processing as described above are performed between an image obtained by photographing the bright pupils 21a and 22a and an image obtained by photographing the dark pupils 21b and 22b at the next time point. After the image is taken, the images of the bright pupils 21c and 22c at the next time point are based on the amount of deviation of the nostrils from 31b and 32b to 31c and 32c between the images of the dark pupils 21b and 22b. Alternatively, position correction and image difference processing may be performed. In this case, the pupil at the position of the image of the bright pupils 21c and 22c is obtained. As described above, when the processing between the images at the adjacent time points is continuously performed without setting only the combination of the image of the bright pupil and the next dark pupil as one unit, the images of the dark pupils 21b and 22b are displayed. After the pupil position is obtained, the pupil position in the next bright pupil image 21c, 22c image and the pupil position in the next dark pupil image can be obtained continuously. If the image and the dark pupil image can be obtained in 1/30 seconds, the position of the pupil can be obtained at 1/60 second intervals.

  Next, a specific example of this embodiment will be described with reference to FIG. In this example, an NTSC camera is used, and a bright pupil is imaged in an odd field and a dark pupil is imaged in an even field. Then, one unit (one frame) is composed of two images of an image obtained by photographing the bright pupil at a certain time point and an image obtained by photographing the dark pupil at the next 1/60 second time point, and is described in FIG. When the image processing is performed to detect the position of the pupil and the processing ends, the next one frame of data already captured in the computer during the processing is subjected to image processing to obtain a new pupil position. The detection process is repeated.

  FIG. 3 shows an overall procedure for analyzing an image. In step S0, the processing for data analysis of the current frame is divided into cases 1 to 4 depending on whether or not the nostril and pupil were detected in the processing of the previous frame. In this example, it does not matter whether both the left and right pupils or nostrils can be detected. If the position of the pupil can be detected as a result of the data analysis of the previous frame, the flag P = 1 in S0, and if the position of the pupil cannot be detected, P = 0. Similarly, if the position of the nostril has not been detected, the flag N = 0, and if it has, N = 1. These flags are identified and are classified into the following cases 1 to 4.

  If neither the nostril nor the pupil is detected in the previous frame, the process of case 1 in step S1 is executed. Since the nostril and the pupil have not been detected at the time of the start of the pupil detection operation, the process of case 1 is executed. The processing procedure of case 1 is shown in FIG. In case 1, in step 11, the pupil is searched from the entire image. When the position of both the nostril and the pupil is unknown, it will be detected from the wide image based on the brightness and size of each image, the positional relationship between the left and right pupils or left and right nostril candidates appearing on the left and right of the screen, etc. However, in the situation where the face hardly moves in 1/60 seconds, the pupil detection can make the pupil stand out by taking the difference between the bright pupil image and the dark pupil image, so that the dark nostril is detected. Is also easy. Therefore, in S11, pupil detection is first performed. In this pupil detection, an image obtained by photographing a bright pupil and an image obtained by photographing a dark pupil are obtained by the method described with reference to FIG. 1, a difference image is formed by subtracting both images, and a binary value is set according to the set luminance. After finding candidates for the left and right pupils, it is determined whether or not they are pupils depending on whether they have predetermined conditions. Needless to say, in the processing of S11, the position correction related to the shift described in FIG. 1 is not performed. Then, in S12, it is determined whether or not the pupil has been detected. If it has been detected, the flag P is set to 1. If not, P = 0 remains and the analysis of the next frame data is started.

  Next, when the pupil can be detected in the previous frame but the nostril cannot be detected, the process of case 2 is performed in step S2. This case corresponds to, for example, a case where a pupil is detected by the processing of case 1. The procedure of case 2 is shown in FIG. Since only the pupil can be detected in the previous frame, and the position of the pupil in the current frame should not have changed much from that, in step S21, the position of the pupil in the previous frame is used as a reference. As shown in FIG. 6, a large window 5 is set in the vicinity where a nostril existing below is predicted, and the nostril is searched in this region. When two left and right pupils are detected in the previous frame, the large window 5 is set downward from an intermediate position between the left and right pupils 21 and 22, as shown in FIG. When only one pupil has been detected, the large window 5 is set based on the detected pupils 21 and 22 and the nostril search is performed as shown in (B) to (D). The nostril search is performed in the same manner for both the bright pupil image and dark pupil image of the current frame. In S22, it is determined whether or not the nostril has been detected. If no detection has been made, the flag N = 0 is maintained and the pupil search is performed in S24. Here, whether or not the nostril could be detected is because it is necessary to have detected the amount of deviation of the nostril as a criterion for correction of deviation when performing the difference between the bright pupil image and the dark pupil image, Nostrils need to be detected in both field images in the current frame. In the pupil search in S24, the position cannot be corrected by the amount of deviation of the nostril, so that the pupil detection without position correction is performed as usual. In this case, since the pupil position is known in the previous frame, the position of the pupil in the current frame can be predicted using a prediction model such as a Kalman filter based on the position of the pupil. Therefore, a small window 6 is set in the predicted portion as shown in FIG. 7, and the pupil is searched for in that region.

  If the nostril can be detected in S21, the flag N = 1 is set in S25, and the pupil search in S26 is performed. The process of S26 is basically the same as that described in FIG. In FIG. 1B, the amount of displacement from the nostrils 31a and 32a in the bright pupil photographed image as the odd field to the nostrils 31b and 32b in the dark pupil photographed image as the even field is obtained. The window 41a, which is predicted to have only the bright pupils 21a and 22a, is corrected to move the window 41a toward the eyelids where the dark pupils 21b and 22b exist as in 41b and 42b, so that the positions of the pupils are matched. After that, the difference between the images is performed to detect the pupil. By the way, in the example illustrated in FIG. 3, when obtaining the pupil position by taking the difference between the images, the bright pupil in the odd field is always moved to the dark pupil side in the even field to obtain the pupil position in the even field. Has been decided. The window predicted to include the pupil in the current frame for determining the region to be analyzed shown in FIG. 1B is because the pupil position of the previous frame used as a reference for prediction is an even field. As for the windows to be predicted and set by the Kalman filter, the windows 41b and 42b in the even field are determined first. Then, the window 41a, 42a in the odd field is set by shifting the amount of elimination of the nostril displacement from the windows 41b, 42b in the even field, and the area of the 41a, 42a is moved to the coordinates of the even field. , 42b, and a difference image is obtained to detect the pupil. If the pupil position cannot be detected in S24 and S26, the flag P = 0 is set in S28, and if it is detected, P = 1 is set in S29 and the process proceeds to the next frame analysis.

  Next, when both the pupil and nostril can be detected in the previous frame, the process of case 3 is performed in step S3. FIG. 8 shows the processing procedure of case 3. In case 3, first, a nostril search is performed in S31. This is because the detection robustness is high if the position of the nostril is known in the previous frame, and if the position of the nostril can be detected first, position correction based on the amount of deviation of the nostril in the subsequent detection of the pupil. This is because the pupil can be searched for. In this case, since the nostril can be detected in the previous frame, the search for the nostril in the current frame is performed using a prediction model such as a Kalman filter based on the position of the nostril in the previous frame as shown in FIG. The position of the nostril in the frame is predicted, the small window 7 is set around the nostrils 31 and 32, and the search is performed in that region. If the nostril position can be detected in the odd field of the current frame, the small window may be set by predicting the nostril position in the next even field using the position data. In S32, it is determined whether or not the nostril has been detected. If not, the flag N is set to 0 in S33, and the pupil is searched in S34. In this case, the position of the nostril cannot be corrected because the nostril position cannot be detected in the current frame, but the position of the pupil in the previous frame is detected, so a prediction model such as a Kalman filter is used based on that value. Thus, the pupil position is predicted, the pupil small window 6 is set in the same manner as in FIG. 7, and the pupil search is performed.

  On the other hand, if it is determined in S32 that the nostril has been detected, N = 1 is set in S35, and the position of the nostril shift similar to S26 is corrected in S36, and the pupil search process using the difference image is performed. If the pupil cannot be detected in S34 and S36, the flag P = 0 is set in S38, and if the pupil is detected, P = 1 is set in S39 to proceed to the analysis of the next frame.

  Next, when the nostril has been detected in the previous frame but the pupil has not been detected, the process of case 4 is performed in step S4. FIG. 10 shows the processing procedure of case 4. In case 4, first, a nostril search is performed in S41. Since the nostril is detected in the previous frame, the nostril search in the current frame is performed using a prediction model such as a Kalman filter based on the nostril position in the previous frame as shown in FIG. The nostril position in the current frame is predicted, the small window 7 is set around the nostrils 31 and 32, and the search is performed in that region. Similarly to S31, when the nostril position can be detected in the odd field of the current frame, the position data may be used to predict the nostril position in the next even field and set a small window. is there. In S42, it is determined whether or not the nostril has been detected. If it has not been detected, the flag N = 0 in S43, and if it has been detected, N = 1 in S44.

  In S45, the pupil is searched. Since the position of the pupil cannot be detected in the previous frame, the position of the nostril detected in the previous frame (or the previous field) is detected as shown in FIG. On the upper side, a large window 8 for the pupil is set, and the pupil is searched in that region. When both nostrils are detected in the previous frame, as shown in FIG. 11 (A), a large window that is symmetric with respect to the positions above the nostrils 31 and 32 and at an intermediate position between the nostrils 31 and 32. Give eight. As shown in FIGS. 11B to 11D, when only one nostril can be detected, the distance between both nostrils is not far from that between both pupils. What is necessary is just to give the large window 8 to the position which becomes. In particular, when the subject is facing sideways as in (D), usually only the nostril 31 in front is detectable, but even in such a case, both pupils are regions of the large window 8 in the same manner. to go into.

  According to this embodiment, it is possible to substantially match the pupil portion in the image obtained by photographing the bright pupil and the image obtained by photographing the dark pupil with reference to the shift amount of the nostril position, and then the difference between the images. I tried to do. Therefore, the head of the target person such as a driver of various transport aircraft or a pupil mouse user of a computer moves within a short time, and there is a time difference in the acquisition of the bright pupil image and the dark pupil image. Even if there is a deviation in the image, in the analysis, the bright pupil and the dark pupil are differentiated in a state where the positions coincide with each other, and are embossed from the surroundings, so that the pupil detection is performed with high accuracy.

  Further, according to the present embodiment, since the position of the nostril is detected and used as a reference for position correction, the position where the pupil has moved is corrected without applying any position reference marker to the face. can do.

  Further, according to the present embodiment, in order to correct the position, the image area to be moved is only the area in which the window is set by a prediction model such as a Kalman filter, and therefore analysis for pupil detection is performed. The image area can be the minimum necessary image area around the pupil, and the calculation efficiency of pupil detection can be improved. In addition, since the area to be analyzed is narrow, the possibility that another image confusing the pupil enters this area can be reduced. Furthermore, the setting of the window based on the prediction model based on the position data of the previous frame or the previous field is also used for pupil detection when position detection by nostril detection or nostril detection cannot be performed. Achieves higher efficiency and higher accuracy in pupil detection work.

  Further, according to the present embodiment, since the image captured earlier in the position correction is moved with respect to the position of the image captured later, the pupil position of the subsequent image is determined based on the pupil position in the latest image. Detection can be performed.

  Further, according to the present embodiment, prior to detecting the amount of deviation of the nostril position, the pupil position is detected by subtracting the bright pupil image from the dark pupil image, and the detected pupil position is used as a reference. A window is set around the nostril, and the nostril is detected by searching the window. At the start of the entire pupil detection operation, it is assumed that the head position is often almost stable. Under such conditions, it is better to detect the pupil using the difference between the images. Since it is easier, the window is set at the position where the position of the nostril is expected based on the position of the pupil that can be detected earlier, and the nostril is searched in that area to improve efficiency. Has achieved.

  In this embodiment, the case where the NTSC system camera is used has been mainly described. However, the camera may be a non-interlace system or the like that is not divided into an odd field and an even field, and in that case, a bright pupil is used for each frame. And an image for shooting the dark pupil. Further, in order to obtain the bright pupil and the dark pupil, means for changing the wavelength of the light source may be used in addition to changing the installation position of the light source. In this case, for example, when light sources having different wavelengths of about 850 nm and about 950 nm are used, the retina reflectance is different. Therefore, an image photographed with a light source of about 850 nm is a bright pupil, and an image photographed with a light source of about 950 nm is dark. It becomes the pupil.

It is a conceptual diagram of the image which image | photographed the bright pupil and dark pupil of embodiment of this invention. It is a conceptual diagram of the image which image | photographed the bright pupil and dark pupil of embodiment of this invention. It is a figure which shows the whole process sequence of embodiment of this invention. It is a figure which shows the one part processing procedure of embodiment of this invention. It is a figure which shows the one part processing procedure of embodiment of this invention. It is a figure which shows a part of image processing of embodiment of this invention. It is a figure which shows a part of image processing of embodiment of this invention. It is a figure which shows the one part processing procedure of embodiment of this invention. It is a figure which shows a part of image processing of embodiment of this invention. It is a figure which shows the one part processing procedure of embodiment of this invention. It is a figure which shows a part of embodiment image processing of this invention. It is explanatory drawing when a pupil image has shifted | deviated. It is explanatory drawing when a pupil image and images, such as a reflection part of glasses, have shifted.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Face, 2 ... Pupil, 21 ... Right pupil, 22 ... Left pupil, 3 ... Nostril, 31 ... Right nostril, 32 ... Left nostril, 4 ... Window, 41 ... Right pupil window, 42 ... Left Pupil window, 5. Large nostril window, 6. Small pupil window, 7. Small nostril window, 8 Large pupil window

Claims (5)

A method for detecting a pupil by subtracting a bright pupil image from a dark pupil image, an image obtained by photographing a bright pupil at a certain time point, and an image obtained by photographing a dark pupil at a subsequent time point or a previous time point A position correction is performed in which an amount of deviation of the nostril position is detected and an image obtained by photographing the bright pupil or an image obtained by photographing the dark pupil is moved in a direction to cancel the deviation by an amount corresponding to the amount of deviation of the nostril position. A method of detecting a pupil by performing and subtracting both images after position correction.   The method for detecting a pupil according to claim 1, wherein the image obtained by photographing the bright pupil or the image obtained by photographing the dark pupil that moves in the position correction is an image of a window region set around the pupil. .   The method for detecting a pupil according to claim 1, wherein the position correction is to move a previously captured image to a position of a later captured image.   Prior to detecting the amount of deviation of the nostril position, a pupil is detected by subtracting a bright pupil image and a dark pupil image, a window is set around the nostril based on the detected pupil position, and the window The method for detecting a pupil according to any one of claims 1 to 3, further comprising a process of detecting a nostril by searching inside. An apparatus for detecting a pupil by subtracting a bright pupil image from a dark pupil image, means for photographing a bright pupil and a dark pupil, an image obtained by photographing a bright pupil at a certain time, and a subsequent time or before Means for detecting a deviation amount of the nostril position in the image obtained by photographing the dark pupil at the time point, and an image obtained by photographing the bright pupil or an image obtained by photographing the dark pupil correspond to the deviation amount of the nostril position. An apparatus for detecting a pupil, comprising: means for correcting a position that moves in a direction that cancels the shift by an amount; and means for detecting a pupil by subtracting both images after position correction.

Images