JP2016051317A - Visual line detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a visual line detection device designed to be able to accurately detect the center of a pupil image and the center of a corneal reflection light and detect a visual line with high accuracy.SOLUTION: A first image-capture condition for accurately acquiring a face image is temporarily determined in a face detection step, and a second image-capture condition, a third image-capture condition, and a fourth image-capture condition in order to be able to accurately acquire a pupil image are temporarily determined in a pupil detection step. A fifth image-capture condition for accurately detecting corneal reflection light is temporarily determined in a corneal reflection light detection step. It is made possible to obtain an accurate image by determining image-capture conditions so as to satisfy all of the first to fifth image-capture conditions.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a gaze detection device that obtains a pupil image from a bright pupil image and a dark pupil image and detects a gaze direction of a subject.

Patent Document 1 describes an invention related to a pupil detection method.
This pupil detection method includes a light source that gives light to the human eye and a half mirror that separates light to be imaged, and transmits the one separated light through a bandpass filter that transmits a wavelength of 850 nm. Obtained by the sensor, and the other separated light is obtained by the second image sensor through the band-pass filter that transmits the wavelength of 950 nm.

  Since the first image sensor can acquire a bright pupil image and the second image sensor can acquire a dark pupil image, the pupil can be detected from the image difference. Further, even when corneal reflection is present inside the pupil part, the corneal reflection can be detected from the dark pupil image, and the line of sight can be detected from the pupil part image and the corneal reflection image. According to the present invention, since the bright pupil image and the dark pupil image are obtained with the same light source turned on, the bright pupil image and the dark pupil image can be acquired at the same timing.

  Also, with this type of pupil detection method, a face image may not be obtained accurately depending on the surrounding light environment such as sunlight, and corneal reflection may not be detected accurately. Therefore, in the pupil detection method described in Patent Document 1, an unilluminated image is acquired in addition to the bright pupil image and the dark pupil image, and the ambient light is obtained by subtracting the unilluminated image from the bright pupil image and the dark pupil image. The influence of the environment is reduced, so that only the bright and dark portions generated by the light source can be taken out.

JP 2008-246004 A

  The pupil detection method described in Patent Document 1 is not easily affected by the surrounding light environment, but it is necessary to use a special camera using a half mirror and two image sensors. The structure becomes complicated. In addition to the bright pupil image and dark pupil image, an unilluminated image must be obtained, which makes it difficult to set image acquisition timing.

  In addition, when the surrounding light environment changes in a complicated manner, such as when mounted on an automobile, it is difficult to continue acquiring accurate images.

  The present invention solves the conventional problems, and provides a line-of-sight detection device that can accurately acquire a pupil image and the like regardless of changes in the surrounding light environment while using the same camera as the conventional one. Objective.

The present invention includes a plurality of light sources that provide detection light to a subject, an imaging member that acquires a face image including an eye image, and a control unit that acquires a bright pupil image and a dark pupil image from the face image. In the line-of-sight detection device,
In the control unit,
(1) measuring the brightness of the face image acquired by the imaging member and obtaining a first imaging condition for keeping the brightness of the face image within a predetermined range;
(2) measuring a luminance of a pupil portion of the bright pupil image, and obtaining a second imaging condition for keeping the luminance of the pupil portion within a predetermined range;
(3) measuring a luminance of a pupil image obtained from a difference between the bright pupil image and the dark pupil image, and obtaining a third imaging condition for keeping the luminance of the pupil image within a predetermined range;
And the imaging condition by the imaging member is adjusted so as to satisfy all of the first imaging condition, the second imaging condition, and the third imaging condition.

  The line-of-sight detection device of the present invention can always acquire an accurate image by obtaining imaging conditions in a plurality of steps.

  Further, the present invention provides (4) a step of measuring the luminance of the corneal reflected light detected from the dark pupil image and obtaining a fourth imaging condition for keeping the luminance of the corneal reflected light within a predetermined range. The imaging condition by the imaging member is adjusted so that the fourth imaging condition is satisfied.

  And (5) obtaining a fifth imaging condition so that at least a part of the image other than the pupil image is negative when the difference between the bright pupil image and the dark pupil image is obtained. The imaging conditions of the imaging member can be adjusted so as to satisfy the fifth imaging condition.

  As described above, by setting the brightness of the dark pupil image to be slightly brighter, when the difference between the bright pupil image and the dark pupil image is obtained, an image other than the pupil image can be made negative. By erasing the minus part from the difference image, the pupil image can always be accurately acquired.

  In the line-of-sight detection apparatus according to the present invention, in the step (2), the second imaging condition is determined so that the luminance of the pupil portion is not saturated so as to be equal to or lower than a first threshold value, and the (3) In step (3), it is preferable that the third imaging condition is determined so as to be equal to or higher than the second threshold value so that the luminance of the pupil image is higher than a certain level.

  As described above, the upper limit of luminance is determined in the bright pupil image acquisition step, and the lower limit of luminance is determined in the pupil image acquisition step, so that the pupil image can be obtained with less influence of the surrounding light environment. become.

  In the line-of-sight detection device of the present invention, the imaging condition is adjusted by at least one of an exposure time and an exposure gain of the imaging member.

  The line-of-sight detection device of the present invention is configured such that when an image is acquired by the imaging member and the imaging condition is obtained from the image brightness, the imaging condition is adjusted at the next image acquisition timing.

  In the present invention, the pupil image and the corneal reflection light can always be accurately detected even if the ambient light environment changes.

The front view which shows an example of arrangement | positioning of the light source and camera of the gaze detection apparatus of embodiment of this invention, Circuit block diagram of the line-of-sight detection device, Explanatory drawing which shows the positional relationship of the direction of the gaze of a human eye and the gaze detection device, An explanatory diagram for calculating the direction of the line of sight from the center of the pupil and the center of the cornea reflection light, Timing chart showing the timing of light source lighting and image acquisition, Operation flow chart of the line-of-sight detection device,

(Configuration of eye gaze detection device)
As shown in FIG. 2, the line-of-sight detection device 1 according to the embodiment of the present invention includes a pair of illumination imaging devices 10 and 20 and an arithmetic control unit CC. As shown in FIGS. 1 and 3, the illumination imaging device 10 and the illumination imaging device 20 are arranged apart by a distance L1.

  Each of the two illumination imaging devices 10 and 20 includes a camera (imaging member) 13, a plurality of first light sources 11, and a plurality of second light sources 12. The optical axis of the illumination imaging device 10 (optical axis of the camera 13) is O1, and the optical axis of the illumination imaging device 20 (optical axis of the camera 13) is O2. The light emission optical axis of the first light source 11 is close to the optical axes O 1 and O 2, and the light emission optical axis of the second light source 12 is farther from the light emission optical axes O 1 and O 2 than the first light source 11. It is in.

  FIG. 3 schematically shows the relative positions of the illumination imaging devices 10 and 20 and the human eye 40. The illumination imaging devices 10 and 20 are installed on an instrument panel, a windshield, or the like, and the optical axis O1 of the illumination imaging device 10 and the optical axis O2 of the illumination imaging device 20 are both directed to the vicinity of the subject's eyes 40. Is set to

  The first light source 11 and the second light source 12 have LEDs (light emitting diodes). The first light source 11 emits infrared light (near infrared light) with a wavelength of 850 nm or a wavelength close to it as the detection light, and the second light source 12 emits infrared light with a wavelength of 940 nm. It is.

  Infrared light (near-infrared light) having a wavelength of 850 nm or a similar wavelength is not easily absorbed by moisture in the eyeball, and the amount of light that reaches and is reflected by the retina located in the back of the eyeball increases. On the other hand, infrared light of 940 nm is easily absorbed by moisture in the eyeball of human eyes. Therefore, the amount of light that reaches the retina located behind the eyeball and is reflected is reduced. Note that light having a wavelength other than 850 nm and 940 nm may be used as the detection light.

  The camera (imaging member) 13 includes an imaging element and a lens. The imaging device is composed of a CMOS (complementary metal oxide semiconductor), a CCD (charge coupled device), or the like. The imaging device acquires a face image including the eyes of the driver that is incident through the lens. In the image sensor, light is detected by a plurality of pixels arranged two-dimensionally.

  The arithmetic control unit CC is composed of a CPU and a memory of a computer, and each block shown in FIG. 2 performs arithmetic by executing software installed in advance.

  The arithmetic control unit CC includes a light source control unit 21, an image acquisition unit 22, a pupil image detection unit 30, a pupil center calculation unit 33, a corneal reflection light center detection unit 34, and a gaze direction calculation unit 35. Yes.

  The light source control unit 21 switches the light emission of the first light source 11 and the light emission of the second light source 12 and controls the light emission times of the first light source 11 and the second light source 12 in each of the illumination imaging devices 10 and 20. And so on.

  The image acquisition unit 22 acquires a face image for each frame in a stereo manner from two cameras (imaging members) provided in the illumination imaging device 10 and the illumination imaging device 20 respectively. The image acquired by the image acquisition unit 22 is read into the pupil image detection unit 30 for each frame. The pupil image detection unit 30 has functions of a bright pupil image detection unit 31 and a dark pupil image detection unit 32. The bright pupil image detection unit 31 detects the bright pupil image, the dark pupil image detection unit 32 acquires the dark pupil image, the difference between the bright pupil image and the dark pupil image is calculated, and the pupil image is displayed brightly. An image is generated. The pupil center calculation unit 33 calculates the center of the pupil image, and the corneal reflection light center detection unit 34 extracts the corneal reflection light from the dark pupil image and calculates the center position thereof. The line-of-sight direction detection unit 35 calculates the line-of-sight direction based on the pupil center and the corneal reflected light center.

  The arithmetic control unit CC is provided with a luminance detection unit 36 and a luminance adjustment unit 37. The luminance detection unit 36 determines the luminance of the face image, the luminance of the pupil included in the bright pupil image, the luminance of the pupil image obtained by the difference between the bright pupil image and the dark pupil image, and the luminance of the corneal reflected light included in the dark pupil image. Are detected. The light source control unit 21 and the image acquisition unit 22 are controlled by the luminance adjustment unit 37 based on the detected values of the respective luminances.

(Bright pupil and dark pupil)
FIG. 3 is a plan view schematically showing the relationship between the direction of the line of sight of the eye 40 of the subject and the illumination imaging devices 10 and 20. FIG. 4 is an explanatory diagram for calculating the direction of the line of sight from the center of the pupil and the center of the corneal reflected light. 3A and 4A show a state in which the line-of-sight direction VL of the subject is directed to the middle between the optical axis O1 of the illumination imaging device 10 and the optical axis O2 of the illumination imaging device 20. 3 (B) and FIG. 4 (B) show a state in which the line-of-sight direction VL is directed toward the optical axis O1.

  The eye 40 has a cornea 41 at the front, and a pupil 42 and a crystalline lens 43 are positioned behind the cornea 41. And the retina 44 exists in the last part.

  Since the detection light having a wavelength of 850 nm is likely to be reflected by reaching the retina 44, when the first light source 11 of the illumination imaging device 10 is turned on, in an image acquired by the camera 13 provided in the same illumination imaging device 10, The infrared light reflected by the retina 44 is detected through the pupil 42, and the pupil 42 appears bright. This image is detected by the bright pupil image detection unit 31 as a bright pupil image. Similarly, in the image acquired by the camera 13 provided in the same illumination imaging device 20 when the first light source 11 of the illumination imaging device 20 is turned on, infrared light reflected by the retina 44 passes through the pupil 42. As a result, the pupil 42 appears bright.

  On the other hand, since the detection light having a wavelength of 940 nm is easily absorbed in the eyeball before reaching the retina 44, the second light source 12 is turned on in any case of the illumination imaging devices 10 and 20. In the image acquired by the camera 13, the infrared light is hardly reflected from the retina 44 and the pupil 42 appears dark. This image is detected by the dark pupil image detection unit 32 as a dark pupil image.

  On the other hand, both the detection light with a wavelength of 850 nm and a wavelength of 940 nm is reflected by the surface of the cornea 41, and the reflected light is detected by both the bright pupil image detection unit 31 and the dark pupil image detection unit 32. In particular, in the dark pupil image detection unit 32, since the image of the pupil 42 is dark, the reflected light reflected from the reflection point 45 of the cornea 41 is detected as a bright spot image.

  In the pupil image detection unit 30, a difference between the dark pupil image detected by the dark pupil image detection unit 32 is obtained from the bright pupil image detected by the bright pupil image detection unit 31, and the pupil image in which the shape of the pupil 42 is brightened. A signal is generated. This pupil image signal is given to the pupil center calculation unit 33. The pupil center calculation unit 33 calculates the center of the pupil 42 by detecting the luminance distribution of the pupil image.

  The dark pupil image signal detected by the dark pupil image detection unit 32 is given to the corneal reflection light center detection unit 34. The dark pupil image signal includes a luminance signal by reflected light reflected from the reflection point 45 of the cornea 41. The reflected light from the reflection point 45 of the cornea 41 forms a Purkinje image, and as shown in FIG. 4, a spot image with a very small area is acquired on the imaging device of the camera 13. The corneal reflection light center detection unit 34 performs image processing on the spot image, and obtains the center of the reflected spot image from the cornea 41 from the luminance portion.

  The pupil center calculation value calculated by the pupil center calculation unit 33 and the corneal reflection light center calculation value calculated by the corneal reflection light center detection unit 34 are given to the gaze direction calculation unit 35. The line-of-sight direction calculation unit 35 detects the direction of the line of sight from the pupil center calculated value and the corneal reflection light center calculated value.

  In FIG. 3A, the line-of-sight direction VL of the human eye 40 is directed to an intermediate portion between the two illumination imaging devices 10 and 20. At this time, the center of the reflection point 45 from the cornea 41 coincides with the center of the pupil 42 as shown in FIG. On the other hand, in FIG. 3B, the line-of-sight direction VL of the human eye 40 is directed in a direction that is biased toward the optical axis O1. At this time, as shown in FIG. 4B, the center of the pupil 42 and the center of the reflection point 45 from the cornea 41 are displaced.

  The line-of-sight direction calculation unit 35 calculates a linear distance α between the center of the pupil 42 and the center of the reflection point 45 from the cornea 41 (FIG. 3B). Further, an XY coordinate having the center of the pupil 42 as an origin is set, and an inclination angle β between the line connecting the center of the pupil 42 and the center of the reflection point 45 and the X axis is calculated. The line-of-sight direction VL is calculated from the linear distance α and the inclination angle β.

  Next, the lighting switching timing of the first light source 11 and the second light source 12 and the timing of the imaging operation of the camera 13 will be described.

  FIG. 5 shows the lighting switching timing of the light sources 11 and 12 in one illumination imaging apparatus 10 and the imaging timing by the camera 13. FIG. 5A shows the lighting timing of the first light source 11 provided in the illumination imaging apparatus 10, and FIG. 5B shows the lighting timing of the second light source 12 provided in the illumination imaging apparatus 10. Yes. (C) shows the exposure time (shutter time) by the camera 13 provided in the illumination imaging apparatus 10.

  In FIG. 5A, when the first light source 11 is turned on at timing T1, an image S1 that is a bright pupil image is acquired by the camera 13, and when the second light source 12 is turned on at timing T2, the dark pupil is obtained by the camera 13. An image S2 to be an image is acquired. Thereafter, when the first light source 11 is turned on at timing T3, the bright pupil image S3 is acquired by the camera 13, and when the second light source 12 is turned on at timing T4, the dark pupil image S4 is obtained by the camera 13. Is taken and this is repeated.

  Also in the other illumination imaging device 20, the bright pupil image and the dark pupil image can be acquired by setting the lighting timing of the first light source 11 and the second light source 12 and the exposure timing of the camera 13.

  By acquiring a bright pupil image and a dark pupil image with each of the two illumination imaging devices 10 and 20, the center of the pupil image and the center of the cornea reflected light are obtained as data on three-dimensional coordinates in a stereo manner. It can be detected.

(Brightness adjustment)
The line-of-sight detection device 1 is one in which the light from the first light source 11 and the second light source 12 is given to the face and the face image is acquired by the camera 13, so the surrounding light environment has a great influence on the detection accuracy. give. For example, if the amount of ambient light is too large, the face image is overexposed, the contours of the facial organs become unclear, and image processing such as cutting out an eye image from the face image becomes difficult. If the light emitted from the first light source 11 is too strong when the amount of ambient light is small, the light guide portion of the bright pupil image becomes light saturated, and the center of the light guide portion cannot be accurately grasped. In addition, if the surrounding light amount is too large, it is not possible to obtain sufficient brightness of the pupil image obtained from the difference between the bright pupil image and the dark pupil image. This also makes it impossible to accurately grasp the shape of the pupil. Furthermore, if there is too much ambient light, it is difficult to accurately grasp the corneal reflection light.

  Therefore, in the arithmetic control unit CC shown in FIG. 1, the luminance detection unit 36 detects the luminance of the bright pupil image obtained from the bright pupil image detection unit 31 and the luminance of the dark pupil image obtained from the dark pupil image detection unit 32. Is done. Based on this detection value, the light source control unit 21 and the image acquisition unit 22 are controlled by the brightness adjustment unit.

  For example, in the time chart of FIG. 5, when the image S1 is acquired by the camera 13 with the first light source 11 turned on at T1, the brightness of this image is measured, and the result is then displayed as the first light source. The image acquisition unit 22 adjusts the imaging conditions for obtaining the image S3 at this time. Further, the lighting time of the first light source 11 at the timing T3 is also adjusted according to the imaging conditions (for example, the exposure time) of the image acquisition unit 22.

  Also, when the second light source 12 is turned on at T2 and the image S2 is acquired by the camera 13, lighting at the time of turning on the second light source 12 at the next timing T4 according to the brightness of the image. Time and imaging conditions for obtaining the image S4 are adjusted. As described above, when the luminance of the image is detected, the result is reflected in the next timing of lighting the same light source, and the lighting time of the light source and the imaging condition are adjusted.

  The imaging condition set and adjusted by the image acquisition unit 22 means at least one of an exposure time (shutter time) by an image sensor provided in the camera 13 and an exposure gain by a gain amplifier.

  Hereinafter, the relationship between the luminance of each image detected by the luminance detection unit 36 and the adjustment control operation by the luminance adjustment unit 37 will be described with reference to the flowchart of FIG.

(A) In ST1 (step 1), a bright pupil image is acquired by the camera 13 when the first light source 11 is turned on, and a dark pupil image is acquired by the camera 13 when the second light source 12 is turned on. . In ST2, a face image including at least eyes is detected from the bright pupil image and the dark pupil image.

  The luminance detection unit 36 detects the luminance of the face image that is a bright pupil image and the face image that is a dark pupil image. At this time, the average brightness of the entire face of the face image of the bright pupil image and the face image of the dark pupil image may be obtained, or the brightness of a region other than the pupil such as a predetermined facial organ, for example, the nose or cheek Alternatively, the average luminance may be obtained. These luminances are compared with a predetermined luminance range value, and the first imaging condition is provisionally determined so that the luminance of the face image of the bright pupil image and the face image of the dark pupil image falls within the luminance range value. (ST2a).

(B) In ST3, the central coordinates of the corneal reflection light are detected by the corneal reflection light center detection unit 34 with respect to the dark pupil image obtained from the dark pupil image detection unit 32. At this time, the dark pupil image obtained by the dark pupil image detection unit 32 is sent to the luminance detection unit 36, and the corneal reflection light (the light from the second light source 12 reflected by the cornea 41 is included in the dark pupil image). Brightness) is detected.

  Here, it is detected whether or not the luminance of the corneal reflection light is saturated, that is, whether or not the luminance of the corneal reflection light is a full scale of the number of bits of image detection data. When the scale is full scale, it is difficult to obtain the center coordinates of the cornea reflected light. In other words, normal cornea-reflected light has a high luminance at the center and decreases as it reaches the periphery, so the center of the cornea-reflected light is obtained by detecting the luminance gradient, but the luminance is saturated. If this is the case, it becomes difficult to determine the center of the corneal reflected light because the luminance gradient cannot be detected.

  Therefore, the luminance adjustment unit 37 temporarily determines the fourth imaging condition for acquiring the dark pupil image so that the luminance of the corneal reflected light is not saturated (ST3a). For example, the brightness of the corneal reflection light is adjusted to be 95% or less of the full scale.

(C) In ST4 of FIG. 6, the pupil image detection unit 30 detects a pupil image from the difference between the bright pupil image and the dark pupil image, and the pupil center calculation unit 33 calculates the center coordinates of the pupil. At this time, in ST4a, the brightness detector 36 detects the brightness of the light guide part of the bright pupil image, and detects whether or not the brightness is saturated, that is, whether or not full scale is reached. When the brightness of the pupil reaches full scale, when the pupil image is obtained from the difference between the bright pupil image and the dark pupil image, there is a high possibility that the pupil image will be saturated, and the luminance gradient is used. The center of the pupil cannot be detected.

  Therefore, the brightness adjustment unit 37 provisionally determines the second imaging condition so that the brightness of the light guide unit in the bright pupil image is less than or equal to the first threshold value in a range where the brightness is not saturated (ST4a). The first threshold value is determined to be about 95% of full scale, for example.

(D) In ST4b, the luminance of the pupil image obtained from the difference between the bright pupil image and the dark pupil image is detected by the luminance detection unit 36, and whether or not the luminance of the pupil image has a brightness that can be identified. Is determined. The brightness adjustment unit 37 determines the third imaging condition so that the detected brightness value is equal to or greater than a predetermined second threshold value. The second threshold value is determined to be about 5% of full scale, for example.
(E) The pupil image is obtained from a difference image obtained by subtracting the dark pupil image from the bright pupil image, but in each part of the face other than the pupil, if the brightness of the dark pupil image is too dark with respect to the bright pupil image, In the difference image, each part of the face image other than the pupil image remains. If the remaining image is clear, there is a problem that the position of the eyes and the position of the pupil cannot be accurately grasped from the difference image.

  Therefore, in ST4c of FIG. 6, the brightness of the bright pupil image and the dark pupil image is detected, and the fifth imaging condition is determined so that the brightness of the dark pupil image is slightly higher. Thus, when the brightness of the dark pupil image is set to be high, when the dark pupil image is minus from the bright pupil image, there are many areas where the difference is minus in the area other than the pupil. In the pupil center calculation unit 33 or the like, it is easy to extract only the pupil image by automatically erasing the negative portion of the difference image as zero.

(F) For example, in FIG. 5, an image S1 that is a bright pupil image and an image S2 that is a dark pupil image are acquired, and the first imaging condition, the second imaging condition, the third imaging condition, and the fourth When the image capturing conditions and the fifth image capturing condition are provisionally determined, the image capturing conditions are determined so as to satisfy all of the first to fifth image capturing conditions, and when the next image S3 and image S4 are detected, Based on the obtained imaging conditions, at least one of the exposure time and exposure gain of the camera 13 is determined, and the lighting times of the light sources 11 and 12 are changed so as to match the exposure time of the camera 13.

(G) In ST5 shown in FIG. 6, since the coordinates of the pupil center and the coordinates of the corneal reflection light center are acquired by the two cameras provided in the illumination imaging devices 10 and 20, each coordinate is expressed in three-dimensional coordinates. Desired. In ST6, a line-of-sight vector is calculated from the coordinates of the pupil center and the coordinates of the corneal reflection light center, and the gazing point is determined in ST7.

  As described above, in the embodiment of the present invention, the luminance detection unit 36 detects the luminance of each image, and the luminance adjustment unit 37 determines the imaging condition, whereby the center of the pupil image can be obtained reliably, and the corneal reflected light is obtained. The center of the can also be reliably detected.

  In the embodiment described above, the imaging condition is determined so as to satisfy all the first to fifth imaging conditions. However, in the present invention, for example, an imaging condition that satisfies all of the second imaging condition, the third imaging condition, and the fourth imaging condition is set, or two conditions of the second imaging condition and the fourth imaging condition are set. The imaging conditions may be determined so as to satisfy all of the combined imaging conditions by combining any required imaging conditions among the five imaging conditions, such as setting the imaging conditions to be satisfied.

  In the above-described embodiment, a bright pupil image is obtained when the first light source 11 is turned on, and a dark pupil image is obtained when the second light source 12 is turned on. However, in other embodiment of this invention, the 1st light source 11 of the illumination imaging device 10 and the 1st light source 11 of the illumination imaging device 20 are made to light alternately, and any 1st light source 11 is set. It is possible to acquire a bright pupil image and a dark pupil image by simultaneously acquiring face images with the camera 13 of the illumination imaging device 10 and the camera 13 of the illumination imaging device 20 when the light is on.

  For example, when the first light source 11 of the illumination imaging apparatus 10 is turned on and a face image is captured by the camera 13 of the illumination imaging apparatus 10, the light from the first light source 11 is reflected by the retina 44 and easily returns to the camera 13. Therefore, a bright pupil image can be acquired. On the other hand, when the first light source 11 of the illumination imaging apparatus 10 is turned on, in the camera 13 of the illumination imaging apparatus 20, the imaging optical axis O2 is a position away from the optical axis of the emitted light. Therefore, a dark pupil image is acquired.

  That is, when the first light source 11 of the illumination imaging apparatus 10 is turned on, a bright pupil image is acquired by the camera 13 of the illumination imaging apparatus 10 and a dark pupil image is acquired by the camera 13 of the illumination imaging apparatus 20. When the first light source 11 of the other illumination imaging device 20 is turned on, a dark pupil image is acquired by the camera 13 of the illumination imaging device 10, and a bright pupil image is acquired by the camera 13 of the illumination imaging device 20.

  Also in this method, by detecting the brightness of the bright pupil image and the dark pupil image, it is possible to optimally set the imaging condition in the same manner as in the above embodiment.

10, 20 Illumination imaging device 11 First light source 12 Second light source 13 Camera (imaging member)
21 light source control unit 22 image acquisition unit 30 pupil image detection unit 31 bright pupil image detection unit 32 dark pupil image detection unit 33 pupil center calculation unit 34 corneal reflection light center detection unit 35 gaze direction calculation unit 36 luminance detection unit 37 luminance adjustment unit 40 Eye 42 Pupil 45 Cornea reflection point O1, O2 Optical axis

Claims (6)

A line of sight provided with a plurality of light sources that provide detection light to the subject, an imaging member that acquires a face image including an eye image, and a control unit that acquires a bright pupil image and a dark pupil image from the face image In the detection device,
In the control unit,
(1) measuring the brightness of the face image acquired by the imaging member and obtaining a first imaging condition for keeping the brightness of the face image within a predetermined range;
(2) measuring a luminance of a pupil portion of the bright pupil image, and obtaining a second imaging condition for keeping the luminance of the pupil portion within a predetermined range;
(3) measuring a luminance of a pupil image obtained from a difference between the bright pupil image and the dark pupil image, and obtaining a third imaging condition for keeping the luminance of the pupil image within a predetermined range;
And the imaging condition by the imaging member is adjusted so that all of the first imaging condition, the second imaging condition, and the third imaging condition are satisfied. (4) measuring the luminance of the corneal reflected light detected from the dark pupil image and obtaining a fourth imaging condition for keeping the luminance of the corneal reflected light within a predetermined range;
The line-of-sight detection device according to claim 1, wherein an imaging condition by the imaging member is adjusted so as to satisfy a fourth imaging condition. (5) obtaining a fifth imaging condition so that at least a part of an image other than the pupil image when the difference between the bright pupil image and the dark pupil image is obtained is negative;
The line-of-sight detection device according to claim 1, wherein an imaging condition by the imaging member is adjusted so as to satisfy a fifth imaging condition. In the step (2), the second imaging condition is determined so as to be equal to or lower than a first threshold value so that the luminance of the pupil is not saturated.
4. The method according to claim 1, wherein in the step (3), the third imaging condition is determined so as to be equal to or higher than a second threshold value so that the luminance of the pupil image is higher than a certain level. The line-of-sight detection device described in 1.   The line-of-sight detection device according to claim 1, wherein the imaging condition is adjusted by at least one of an exposure time and an exposure gain of the imaging member.   The line-of-sight detection device according to claim 1, wherein when an image is acquired by the imaging member and the imaging condition is obtained from the image brightness, the imaging condition is adjusted at a next image acquisition timing.