JPS6177705A - Device for recognizing position of vehicle driver's eyes - Google Patents

Abstract

PURPOSE:To make it possible to recognize the three-dimensional positions of the driver's eyes simply and accurately, by using two kinds of image data obtained based on the different light projecting directions, and detecting the positions of special points and the positions of the eyes. CONSTITUTION:The two special points on the face part of a vehicle driver I are picked up by an image detecting means III. Thus the two kinds of image data are obtained. For this purpose, a light emitting means II is composed of two light emitting parts V, which are arranged at different positions. In order to emit light from the light emitting parts V, individually, an emission control means VI, which outputs an emission control signal, is provided for the means II. With the two kinds of the image data detected by the means III as the first and second special-point detecting data, an image processing means IV detects the position of the first special point in the image data based on the first data. With the detected position as a parameter, the position of the second special point in the image data is detected based on the second data. The three-dimensional positions of the eyes of the driver I in the vehicle are obtained based on the positions of the detected first and second special points.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a vehicle driver's eye position recognition device that recognizes the position of the vehicle driver's eyes. The present invention relates to a vehicle driver's eye position recognition device that recognizes the three-dimensional position of the driver's eyes using two types of image data obtained from a vehicle driver.

[Prior art] In recent years, with the rapid development of electronic devices, research has been progressing on new devices aimed at improving vehicle operability, comfort, accident avoidance, etc. position,
In particular, it recognizes the position of the eyes and automatically adjusts the angle of the rearview mirror, air conditioning air outlet, or steering wheel according to the position of the eyes, and detects drowsiness while driving based on periodic changes in the position of the eyes. The idea is to give warnings to people.

Conventionally, as one of the methods for recognizing the position of the vehicle driver's eyes used to execute such control,
The facial part of the vehicle driver is captured as image data, the part of the image data that is most similar to a predetermined standard eye pattern is detected as the eye position, and the driver's eye position is determined from the eye position in this image data. A device designed to find five-dimensional positions is being considered.

Furthermore, if the eye position is directly detected by simply comparing the image data with the eye standard pattern as described above, there may be places other than the eye position that are similar to the eye standard pattern, leading to incorrect eye positions. It is possible to detect
In order to improve the detection ability, we first need to find a location on the image that is easier to detect, that is, for example, when the image data is a side image of the driver's face, the tip of the nose that protrudes from the face (
Hereafter, it will be referred to as advantages and disadvantages. ) as the singular point position, then estimate the eye position in the image data from this singular point position, and set the position of the eye where the image data in the vicinity is most similar to the standard pattern of the eye. It is considered.

By the way, if you detect the position of the singular point on the image data as described above and detect the position of the eye based on the position of the singular point, then both the position of the singular point and the position of the eye will be detected in the image data. By using image data that is easy to use, it becomes possible to detect the position of the eyes more accurately.For example, as shown in Figure 2, it is possible to obtain image data that makes it easy to detect the position of the nose, which is a singular point. If the lighting is applied in a way that makes it difficult to detect the eyes, shadows may be created near the eyes, and if the lighting is applied in such a way that the positions of the eyes can be easily detected, as shown in Figure 3. This results in problems such as a shadow being formed in the nose area, resulting in image data that makes it difficult to detect the position of the tip of the nose.

[Object of the Invention] Therefore, the present invention has been made in view of the above points, and is configured to detect the singular point position and the eye position using two types of image data obtained by different irradiation directions. The configuration of the present invention for achieving the above object is as shown in FIG. As shown in , a light emitting means (2) for illuminating the vehicle driver (2), and an image detection means (3) for receiving reflected light from the driver (2) caused by the illumination by the light emitting means (2) and detecting at least the facial part of the driver (2) as image data. A device for recognizing the position of the eyes of a vehicle driver, comprising: (1) arithmetic processing of the detected image data to determine the three-dimensional position of the eyes of the driver; In the means (2), the light emitting means (2) are arranged at different positions so that two different types of image data in which two predetermined singular points on the face of the vehicle driver are clearly imaged are obtained. In addition, an irradiation control means (I) is provided for outputting an irradiation I control signal to the light emitting means (■) in order to cause the two light emitting parts (iv) to emit light individually, The processing means (■) processes one of the two different types of image data detected by the image detection means (■) into the first one.
the singular point position detection image data, and the other as the singular point position detection image data, and detecting the first singular point position in the image data from the first singular point position detection image data, The detected first singular point position is 1
The second singular point position in the image data is detected from the image data for detecting the second singular point position as one parameter, and the detected second singular point position or the first singular point position is detected from the second singular point position detection image data.
The gist of the present invention is a vehicle driver's eye position recognition device, characterized in that the device is configured to obtain the three-dimensional position of the eyes of the driver I in the vehicle from the singularity position and the second singularity position. .

[Example] An embodiment of the present invention will be described below with reference to the drawings.

FIG. 4 is a perspective view showing the instrument panel part and its surroundings of a vehicle in which the eye position recognition IA system of this embodiment is installed, where 1 is the driver's seat, 2 is the driver's seat, and 3 is the instrument panel. It represents. Two light emitting units 5 and 6 are provided at diagonally to the left front of the instrument panel 3 with respect to the driver's seat 2, that is, in front of the passenger seat side, and are provided at different positions to illuminate the upper body of the driver 1 from the left direction. and,
An image detection section 7 is provided that receives reflected light from the driver 1 due to the irradiation of the light emitting section 5 or 6 and captures a two-dimensional image of the upper body of the driver 1 from the left direction.

Here, the light emitting units 5 and 6 constitute the above-mentioned light emitting means (2), and an infrared strobe that emits infrared light is used so as not to dazzle the driver 1 when irradiating the light. As shown in Fig. 5, an infrared light emitter 5a (6a>), a lens 5b (6b) for broadly irradiating the driver with infrared light emitted by the infrared light emitter 5a (6a), and an infrared light emitter 5a (6a). An infrared filter 5c (6c) that transmits light but does not transmit visible light, an infrared emitter 5a (6a), a lens 5b (6b), and an infrared filter 5c (6c) are stored in the instrument panel 3. Case 5d (6
d), and are arranged on the left and right sides of the image detection section 7 in this embodiment.

The image detecting section 7 corresponds to the above-mentioned image detecting means (2), and is for capturing the light emitted from the driver 1 by the light emitting section 5 or 6 as an image of the upper body of the driver. As shown in FIG. 6, the detection unit 7 includes an infrared filter 7a for transmitting reflected light from the driver 1, that is, infrared light, and a solid-state image carrier 7e for transmitting an image of the driver's 1's body. A lens 7b having a focal length of, for example, 12.5 mm to form an image on the glenoid plane, a liquid crystal aperture element 7C for adjusting the amount of light, and a phototransistor 7d for detecting the light I transmitted through the liquid crystal aperture element 7C. , the infrared filter 7a, and the lens 7b.
and a MOS-type solid-state ms element 7e consisting of a photosensor and a switching circuit, which extracts the image formed on the imaging surface transmitted through the liquid crystal aperture element 7C as an electrical signal by switching scanning, and each of the above parts is stored therein. A solid-state camera is used, which includes a case 7f for attachment to the instrument panel 3.

Next, the entire groove of the eye position recognition device of this embodiment will be explained based on the block diagram shown in FIG.

As shown in the figure, book g f! The device includes the two light emitting parts 5.
6 and the image detecting section 7, the light emitting sections 5 and 6 are individually arranged so as to recognize the position of the eyes of the driver 1 from the two types of image data as shown in FIGS. 2 and 3. At the same time as emitting light, processing the image signal obtained by the image & detection unit 7,
It includes an image processing section 8 that recognizes the position of the driver's 1 eyes. The image processing section 8 controls the image detection section 7, converts the image signal obtained by the image detection section 7 into a digital signal, and temporarily stores the converted digital signal as image data. an irradiation signal output section 12 that outputs an irradiation signal for causing the light emitting sections 5 and 6 to emit light alternately; A central processing unit (C
PU) 14, a read-only memory (ROM) 16 in which control programs and data used in executing recognition processing are stored in advance, and data used in executing arithmetic processing, etc. a random access memory (RAM) 18 in which the data is temporarily read and written;
A path line 20 that connects each of the above parts and serves as a path for image signals and control signals, and a power supply circuit 22 that supplies power to each of the above parts.
It is composed of. Further, the image signal input section 10 receives an image signal from the image detection section 7 and a synchronization signal generation circuit 10a that outputs a vertical synchronization signal and a horizontal synchronization signal to the image detection section 7 consisting of the solid-state camera, and receives an image signal from the image detection section 7, for example, for one frame. Image data for two frames converted by the A/○ conversion circuit 10b and the A/D conversion circuit 10b, which converts into image data consisting of digital signals representing the brightness of 256 pixels horizontally and 240 vertically. , that is, an image data storage circuit 10c that stores two different types of image data obtained by light emission from the light emitting section 5.6.

Next, the operation of the eye position recognition process executed by the image processing section 8 will be described in detail with reference to the flowchart shown in FIG. Note that this process is executed when a request for recognition of the position of the driver's 1 eyes is input by a signal of a fixed period, a signal from a switch, or the like.

As shown in the figure, when the process is started, step 101 is first executed, and by emitting light from the light emitting units 5 and 6 individually, two types of images are captured by the image detecting unit 7, namely, FIGS. 9 and 10. An image data reading process is executed in which two types of images with different irradiation directions as shown in the figure are stored as two types of image data consisting of digital signals in the image signal input section 1o.

Here, the image signal input section 10 is a synchronization signal generation circuit 10a.
1A/D conversion circuit 10b and image data storage circuit 10c
It controls the image detection section 7 according to the vertical synchronization signal and horizontal synchronization signal output from the synchronization signal generation circuit 1Qa, sequentially captures the image signals output from the image detection section 7, and converts the A/D conversion circuit. Since the image signal converted into a digital signal via 10b can be stored as image data in the image data storage circuit 10C, the main image data reading process involves activating the image signal input section and transmitting the synchronizing signal generating circuit 10a. This can be carried out by causing the light emitting sections 5 and 6 to individually emit light in synchronization with the vertical synchronization signal output from the vertical synchronization signal. That is, as shown in FIG. 11, first, the delay time ΔT! from the rising timing to of the vertical synchronizing signal is such that the light emission of the light emitting unit 5 coincides with the vertical retrace timing t1! !
By outputting the after-light irradiation signal, the image signal input section 10 displays the second image shown in FIG. 9 obtained by the image detection section 7 during the time ΔT1. Then, by outputting the irradiation signal S2 so that the light emission of the light emitting unit 6 coincides with the next vertical retrace timing t2, the image signal input unit generates an image as shown in FIG. 10 during the time ΔT2. will be read as image data G2. In addition, the image data G+' (or G2) that is A/D converted and stored in this image signal input section is a pixel that divides the image obtained by the image detection section 7 into 256 horizontally and 240 vertically. The structure can be expressed as shown in FIG. 12.

In this way, the light emitting units 5 and 6 emit light in step 101, and the two types of image data G1,
When G2 is read, the following step 102 is executed, and among the two types of image data read above, image data G1 shown in FIG. 9 in which the nose of the driver 1 is clearly displayed is A predetermined area, that is, 30 areas shown in Fig. 9, is read as the area where the nose exists, and a pixel that exists on the leftmost side of this area and has a brightness above a certain level is searched (hereinafter referred to as bright spot search). That's what it means.

) and remember its position. This process is carried out in the next step 10.
When detecting the position of the nose in step 3, this is a process for limiting the range in which the nose is searched, and the determined pixel position is used as the base point for searching the nose. The bright spot search process is as follows:
This can be carried out by sequentially searching in the vertical direction starting from the upper left pixel of the nose position 30 in FIG. 9, and storing the position of the first pixel having brightness above a certain level.

When the bright spot search is executed in this way and the position of the pixel that exists on the leftmost side of the nose position in the image data G1 and has a brightness of a certain level or higher is determined, in the subsequent step 103 as described above, this position is determined. Processing is performed to search for the pixel using the pixel as a base point and determine the outer position. The degree of correlation between the standard pattern and the image data G1 is checked, and the position of the image where the correlation is maximum is regarded as the position where the nose exists, and the position of the nose is determined.

In other words, centering on the pixel position Gt (XSV) obtained in step 102 above, as shown in FIG. 14, there are 5 pixels each on the top, bottom, left and right, for a total of 121 pixels, as shown in FIG. 13. Center position in standard nose pattern +11 (0
, 0), the correlation value between the image data G1 and the standard pattern is calculated, and the pixel position where the value is the largest is obtained or lost (
The so-called tip of the nose is recognized as WG (hx, hy). Here, the center position m(0,0) in the standard nose pattern shown in FIG. 10 is preset as a portion corresponding to advantages and disadvantages, and the phase r! jJl111 can be obtained from the following formula (% formula %). As for the values of i and j above, the standard pattern of the nose is configured as data with a spread of 16 above, 7 below, 1 on the left, and 6 on the right with respect to the center m (0, 0). For i, from (-1) to (+
6), an integer value between (-16> and (+7)) may be used for j.

Also, the correlation value is the center position R1 in the standard pattern of the nose.
(0,0), image data G as shown in FIG.
Since the calculation is performed by applying a total of 121 pixels, 5 each on the top, bottom, left and right of pixel G1 (X, V) above 1, the value of X1Y in the above equation is ±5, Y becomes an integer value between y±5, and eventually X and Y
A maximum of 121 types of correlation values are calculated by the combination of , and the pixel G+(X,,Y
) is detected as the nasal position flG+(hx, hy).

In this way, the nose position G1 (hx
Once ``lhy'' is obtained, the process moves to the following step 104, and this time, this nose position G+ (hx, hy) is converted to the pixel position G 2 (hx, hy) on the image data G2.
The position of the eye is detected using the image data G2 shown in FIG. 10, in which the eye clearly appears. To find the position of the eyes, first the position of the nose G2
Find the pixel position @Gz (hx + 15, hy-21) that is 21 pixels up 15 pixels to the right from (hx, hy), and select the eye as shown in Figure 15 that has been set in advance near this position. The degree of correlation between the standard pattern and the image data is examined to determine the eye position G(IIIX, ay>.This determination method can be performed in the same manner as the nose position determination in step 103 above, so the explanation will be given below.) is omitted.

In the above processing of steps 102 to 104, the eye position G 2 (IIIX, ID
Once V) is determined, the process moves to step 105, where the three-dimensional position of the driver's 1 eyes within the vehicle interior is determined.

In this embodiment, since the image detection unit 6 is fixed, the eye position G 2 (mX, mV) obtained in step 104 is used, and the driver 1 does not move in the left-right direction. As a result, the three-dimensional position of the eyes is required. In other words, the left and right (Z
) direction, and set the standard position P @ standard position P @ standard three-dimensional position 11M of driver 1's eye (left eye) as the coordinate (
The location of the block can be recognized by determining the image data G as
The position G2 (9X, I)Y) of the standard point P on 2 is memorized in advance, and this reference position Gz (1)X, I)
The position f for V)! t G 2 (l1xs mV
) can be easily carried out by calculating the deviation.

As explained above, in the eye position recognition device of this embodiment, the two light emitting units 5 and 6 illuminate the driver 1 from different directions, and the reflected light from the driver 1 due to the irradiation is detected as an image. By providing the image detection unit 7 on the diagonally front part of the instrument panel 3 with respect to the driver's seat 2, two types of image data G1 are obtained in which the nose and eyes of the driver's face are clearly captured. and G2, and first, from the image data G1, 0 of the driver 1 is obtained.
After detecting the cusp position as the first singular point position and detecting the eye position, which is the second singular point position, from the image data G2 using this first singular point position as one parameter, the eye position of the driver 1 is detected. It is designed to recognize the three-dimensional position of. Therefore, the position of the nose and the position of the eyes, which are two singular points on the image data, can be detected with high precision without erroneously detecting them, and the three-dimensional position of the eyes can be recognized more accurately. Therefore, the eye position recognition device of this embodiment can be used to automatically adjust the angles of rearview mirrors, air-conditioned air outlets, steering wheels, etc., or to detect drowsy driving and issue a warning to the driver. These controls can be executed more precisely, improving vehicle operability, comfort,
Safety can be further improved.

Here, in this embodiment, the image processing section 8 that executes the control shown in FIG. corresponds to the irradiation control means (2), and the processing of steps 102 to 105 corresponds to the image processing means V.

In this embodiment, the image detection unit 7 is a MOS type solid @
The three-dimensional position of the eyes was recognized using a single so-called solid-state camera consisting of a quadrilateral element. If it is desired to recognize the position of the eyes with higher accuracy by taking into consideration the driver's movement in the left and right directions, two image detection units are provided, and the above processing is executed for each image detection unit.
The three-dimensional position of the driver's eyes may be recognized from the position of the eyes on each image data using the principle of so-called triangulation. Furthermore, as the image detection section 7, a MOS type solid state 11 is used.
[! In addition to a solid-state camera made of a solid-state image sensor, a solid-state camera made of another solid-state image sensor such as a COD may also be used.
A 1ffue tube can also be used.

Furthermore, in this embodiment, the position of the driver's eyes on the image data G2 is detected as the second singular point position, and the three-dimensional position of the eyes is calculated. If you are wearing glasses or sunglasses, it is possible that the position of your eyes cannot be detected well, so in this case, for example, the position of your jaw is detected as the second singular point position, and If the position of the driver's eyes is calculated from average human data using the position of Kotobuki, which is the singularity position of No. 1, as a parameter, the position of the eyes of the driver can be recognized better. .

[Effects of the Invention] As described in detail above, in the eye position recognition device of the present invention, the light emitting means is constituted by two light emitting sections disposed at different positions, and the light emitting sections are configured to clearly emit light. Of the two types of image data obtained by this, the image data in which the first singular point is clearly mV& is used as the image data for detecting the position of the first holding tool point, and the image data in which the second singular point is clearly shown in the b1 image. The first singular point position is detected using the first singular point position detection image data as the second singular point position detection image data, and then this first singular point position is detected. After detecting the second singular point position using the image data for detecting the second singular point position as one parameter, from the second singular point position or from the first singular point position and the second singular point The three-dimensional position of the vehicle driver's eyes is calculated from the position. Therefore, the positions of two singular points on the image data can be detected with high accuracy without erroneously detecting them, and the three-dimensional position of the eye can be accurately determined. Therefore, when using the eye position riiVft position of the present invention to perform controls such as automatically adjusting the angle of the rearview mirror, air conditioner air outlet, steering wheel, etc., or detecting drowsy driving and issuing a warning to the driver, , these controls can be executed more precisely, and the operability, comfort, and safety of the vehicle can be further improved.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of the present invention, FIGS. 2 and 3 are explanatory diagrams showing images of the driver's face and explaining the conventional problems, and FIGS. 4 to 16 are diagrams showing the structure of the present invention. 4 is a perspective view showing the vehicle instrument panel and its surroundings, FIG. 5 is a sectional view showing the configuration of the light emitting section 5 or 6, and FIG. 6 is the structure of the image detection section 7. 7 is a block diagram showing the overall configuration of this embodiment, FIG. 8 is a flowchart explaining the operation of the image processing section 8, and FIG. An image data diagram explained in terms of density, FIG. 10 is an image data diagram similarly obtained by irradiation with the light emitting unit 6, FIG. 11 is a time chart explaining the operation of each part by image data reading processing, and FIG. 12 is an image data diagram. Data diagram representing image data stored in the data storage circuit 10c, 1st
Figure 3 is a data diagram showing the standard pattern of the nose, Figure 14 is an explanatory diagram showing the calculation range of correlation values in nose position detection processing and eye position detection processing, and Figure 15 is a data diagram showing the standard pattern of eyes. , FIG. 16 is an explanatory diagram illustrating a method for calculating the three-dimensional position of the eyes. ■... Light emitting means ■... Image detecting means ■... Image processing means V, 5.6... Light emitting section Vl... Irradiation control means 7... Image detecting section 8... Image processing Section 10... Image signal input section 14... CPU

Claims (1)

[Scope of Claims] 1. A light emitting means for illuminating a vehicle driver; and an image detecting means for receiving reflected light from the driver due to the irradiation of the light emitting means and detecting at least the face of the driver as image data. , an image processing means for calculating the three-dimensional position of the eyes of the driver by calculating the detected image data, the apparatus comprising: an image processing means for calculating the three-dimensional position of the eyes of the driver; In order to obtain two different types of image data in which two predetermined singular points on the face of the vehicle driver are clearly imaged, the light emitting means is arranged at two different positions. comprising a light emitting section, and further comprising an irradiation control means for outputting an irradiation control signal to the light emitting means in order to cause the two light emitting sections to emit light individually, and further detecting the image processing means with the image detection means. Of the two different types of image data, one is the first image data for singular point position detection and the other is the second image data for singular point position detection, and from the first image data for singular point position detection The first singular point position in the image data is detected, and the second singular point position in the image data is detected from the second singular point position detection image data using the detected first singular point position as one parameter. configured to detect a point position and obtain a three-dimensional position of the driver's eyes in the vehicle from the detected second singular point position or the first singular point position and the second singular point position. A vehicle driver's eye position recognition device characterized by: 2. The vehicle driver's eye position recognition device according to claim 1, wherein the first singular point position is the position of the driver's nose. 3. Claim 1, wherein the second singular point position is the position of the driver's eyes, and the image processing means is configured to determine the three-dimensional position of the vehicle driver's eyes from the second singular point position. Or the vehicle driver's eye position recognition device according to item 2. 4. Vehicle operation according to any one of claims 1 to 3, wherein the image detection means is provided diagonally in front of the driver's seat, and the two light emitting parts are provided on the left and right sides of the image detection means, respectively. Human eye position recognition device. 5. The vehicle driver's eye position recognition device according to any one of claims 1 to 4, wherein the image detection means includes a two-dimensional solid-state image sensor. 6. The vehicle driver's eye position recognition device according to any one of claims 1 to 5, wherein the light emitting means is configured to emit infrared light.