JPS6177703A - Device for recognizing positions of vehicle driver's eyes - Google Patents

Abstract

PURPOSE:To made the recognition of the position of the eyes even if glasses and the like are worn, and to make it possible to improve accuracy by recognizing the position of the eyes based on the position of the nose tip and the inclination of a straight line connecting the nose tip and the nose root on image data, which is obtained by detecting the light reflected by the driver. CONSTITUTION:An image detecting means III receives the light reflected from a driver I based on the projection of light from a light projecting means II and detects at least the surface part of the face as the image data. The position of the nose tip and the shape of the nose back in the detected data are detected by a nose-tip-position detecting means IV and a nose-back-shape detecting means V. an inclination operating means VI computes the inclination of a straight line connecting the nose tip and the nose root based on the detected nose tip position and the nose back shape. The position of the driver's eyes in a vehicle is computed by an eye-position computing means VII based on the computed inclination and the nose tip position.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a vehicle driver's eye position recognition device that recognizes the position of the vehicle driver's eyes, and in particular detects the driver as an image and detects the tip of the nose, The present invention relates to a vehicle driver's eye position recognition device that recognizes the position of the driver's eyes based on the slope of a straight line connecting the tip of the nose and the root of the nose.

[Conventional technology] With the rapid development of electronic equipment in recent years, research has been progressing on new devices aimed at improving vehicle operability, comfort, accident avoidance, etc. It recognizes the position, especially the position of the eyes, and automatically adjusts the angle of the rearview mirror, the air outlet, or the steering wheel.
Consideration is being given to detecting drowsy driving and issuing a warning to the driver.

Conventionally, as an eye position recognition device for recognizing the position of the driver's eyes used for the above-mentioned control, for example, image detection units that capture the driver's upper body as image data are installed at two locations in front of the driver's seat. The two horizontal black parts of the face in the image data are estimated as the positions of the eyes, and the three-dimensional position of the eyes is recognized using the principle of triangulation. An image detection unit is installed at the location, and a protrusion on the face, that is, the driver's nose, is detected as a singular point position in the image data obtained by the image detection unit, and based on this singular point position, the left and right direction of the driver is detected. One idea is to recognize the three-dimensional position of the eyes assuming that there is no movement.

[Problems to be Solved by the Invention] In the case of the above-mentioned former recognition device, a problem arises in that the position of the eyes cannot be recognized well when the driver uses visual inspection or sunglasses, etc. On the other hand, in the case of the latter recognition device, since the eye position is simply estimated from one singular point position, the error is large and it is difficult to accurately recognize the eye position.

Therefore, the present invention has been made to solve the above problem, and it is possible to calculate the position of the driver's nose from the tip of the driver's nose in the image data obtained by the image detection unit, the slope of the straight line connecting the nose tip and the root of the nose, and the position of the eyes. The purpose is to make it possible to recognize the exact position of the eyes by estimating the position of the eyes.

[Configuration of the Invention] As shown in FIG. 1, the configuration of the present invention to achieve the above object includes a light emitting means (1) for illuminating the vehicle driver, and a light reflected from the driver by the illumination of the light emitting means. an image detection means surface for detecting at least the facial part of the driver as image data; a nose tip position detection means (2) for detecting the position of advantage or disadvantage of the driver in the detected image data; and the detected image data. nasal dorsum shape detection means V for detecting the shape of the nose dorsum of the driver; and slope calculation means Vl for calculating the inclination of a straight line connecting the nasal tip and the nasal root based on the detected advantages and disadvantages positions and the nasal dorsum shape. and an eye position calculation means (2) for calculating the eye position of the driver in the vehicle based on the detected inclination and the peak position. The gist is the number @ recognition device U.

[Example] An embodiment of the present invention will be described below with reference to the drawings.

FIG. 2 is a perspective view showing the instrument panel part and its surroundings of a vehicle in which the eye position recognition device of this embodiment is installed, in which 1 is the driver's seat, 2 is the driver's seat, and 3 is the instrument panel. It represents. A light emitting section 5 that illuminates the upper body of the driver 1 from the left side is located diagonally forward to the left of the driver's seat 2 of the instrument panel 3, that is, in front of the passenger seat side. An image detection unit 6 is provided which receives the reflected light and captures a two-dimensional image of the upper body of the driver 1 from the left direction.

Here, the light emitting unit 5 corresponds to the above-mentioned light emitting means (2), and an infrared strobe that emits infrared light is used so as not to dazzle the driver 1 during irradiation, as shown in FIG.
An infrared light emitter 5a, a lens 5b for broadly irradiating the driver with infrared light emitted by the infrared light emitter 5a, and an infrared filter 5C that transmits infrared light but blocks visible light. , a case 5d for storing the infrared light emitter 5a, lens 5b, and infrared filter 5C and for attaching it to a predetermined position on the instrument panel 3.

Further, the image detection section 6 corresponds to the above-mentioned image detection means (2), and is for capturing the reflected light from the driver 1 due to the irradiation of the light emitting section 5 as an image of the driver's 1 body. 6, as shown in FIG. 4, reflected light from the driver 1;
That is, an infrared filter 6a for transmitting infrared light, and a lens 6 having a focal length of, for example, 12.5 mm for forming an image of the body of the driver 1 on an imaging surface of a solid-state image sensor 6e, which will be described later.
b, a liquid crystal diaphragm element 6C for adjusting the amount of light, and an image formed on the imaging surface transmitted through the infrared filter 5a, lens 6b, and liquid crystal diaphragm element 6C, which is taken out as an electrical signal by switching scanning. A non-interlaced solid-state camera is used, which consists of a MOS-type solid-state dynamic element 6e consisting of a photosensor and a switching circuit, and a case 6f for storing the above-mentioned parts and attaching it to the instrument panel 3. ing.

Next, the overall configuration of the eye position recognition device of this embodiment will be explained based on the block diagram shown in FIG. 5.

As shown in the figure, in addition to the light emitting unit 5 and image detecting unit 6, this recognition device processes image signals obtained by the image detecting unit 6, and performs image signal processing to recognize the position of the eyes of the driver 1. It is equipped with part 8. The image signal processing section 8 controls the image detection section 6, converts the image signal obtained by the image detection section 6 into a digital signal, and temporarily stores the converted digital signal as image data. an irradiation signal output section 12 that outputs an irradiation signal for causing the light emitting section 5 to emit light; and an irradiation signal output section 12 that outputs an irradiation signal for causing the light emitting section 5 to emit light. A central processing unit (CPIJ) 14 executes a series of eye position recognition processes such as detecting the tip of the nose and the shape of the nose dorsum of the driver 1 and recognizing the positions of the eyes of the driver 1 from these two, and the CPU 14. a read-only memory (ROM) 16 in which III I programs and data for executing eye position recognition processing are stored in advance;
A random access memory (RAM) 18 in which data used for executing arithmetic processing is temporarily read and written, a path line 20 that connects each of the above parts and serves as a path for image signals and control signals, and a power supply to each of the above parts. Supply power circuit 2
It is composed of 2.

Next, the eye position recognition processing performed by the CPtJ14 of the image signal processing section 8 will be described in detail with reference to the flowchart shown in FIG.

When the process is started, first in step 101, a light emission process is executed in which the irradiation signal output section 12 outputs an irradiation signal in order to cause the light emitting section 5 to emit light.

This process is carried out by the synchronization signal generation circuit 10a of the image data input section.
This is to cause the light emitting unit 5 to emit light in synchronization with the vertical synchronization signal output from the vertical synchronization signal, so that good image data can be obtained at the image data input unit, as shown in FIG.
The irradiation signal is output after a predetermined delay time ΔT has elapsed from the rise time of the vertical synchronization signal so that the light emission of the light emitting unit 5 coincides with the vertical retrace time t1. When the driver 1 is irradiated with infrared light from the light emitting unit 5 in this way, the image signal from the image detection unit 6 is converted into a digital signal by the image data input unit 10 during time t2, and the digital signal is will be stored as image data in the image data storage circuit 10C.

In this image data storage circuit 10c, the light emitting section 5 and the image detecting section 6 are connected to the driver's seat 2. 8, the image of the driver 1 taken diagonally to the left turns as shown in the ninth factor,
Image data representing the brightness of pixels divided into 256 pixels horizontally and 240 pixels vertically is stored as G(○, O)...G(255,239>).

Next, in step 102, a predetermined area set in advance in the image data, that is, 3 as shown in FIG.
The area of 0 is read as the area where the nose exists, and the leftmost pixel in this area that has a brightness above a certain level is searched (hereinafter referred to as bright spot search) and its position is stored. This process is a process for limiting the range to search for the nose when detecting the position of the nose in the next step 103.
The position of the determined pixel is used as the base point for searching for the nose.

In this bright spot search process, starting from the upper left pixel of the nose position 30 in FIG. Just remember it.

When the bright spot search is executed in this way and the position of the pixel that exists on the leftmost side of the nose position and has a brightness above a certain level is determined, in the subsequent step 103 as described above,
Processing is performed to search for the nose using this position as a base point and determine the position of the nose.
The degree of correlation between the standard pattern of the nose as shown in the figure and the image data is examined, and the position of the image where the correlation is maximum is regarded as the position where the nose exists, and the position of the nose is determined.

In other words, centering on the pixel position G (x, y) determined in step 102 above, as shown in FIG. 11, 5 pixels each on the top, bottom, left and right, totaling 121 m of pixels, as shown in FIG. Center position l in the standard nose pattern
Calculate the correlation value between the image data and the standard pattern as (○, O), and set the pixel position where the value is the largest to the tip of the letter (
It is recognized as the position t3 (hx, hy) of the so-called 0 head). Here, the center position m (0, O) in the standard nasal pattern shown in FIG. 10 is set in advance as a portion corresponding to the tip of the nose, and the correlation value can be obtained from the following equation (1). The values of i and j above are configured as data in which the standard pattern of the nose extends from the center m (0, ○) by 16 above, 7 below, 1 to the left, and 6 to the right. Therefore, i has (-1) to (+6
>, an integer value between (-16> and (+7)) may be used for j.

Also, the correlation value is the center position m in the standard pattern of the nose.
(0,0) is calculated by applying a total of 121 pixels, 5 each to the top, bottom, left and right, centering on the pixel G (x, y>) on the image data as shown in Figure 11. Therefore, the values of X and Y in the above equation are
is an integer value between X±5 and Y is an integer value between y±5, and in the end, a maximum of 121 types of correlation values are calculated by combining X and Y.
The pixel G (X, Y) on the pixel data having the maximum value is detected as the gain/loss position G (hx, hy).

Once the advantage/disadvantage position G (hx, by) is determined in this way, the process moves to the subsequent step 104, where the shape of the dorsal dorsum (so-called nose bridge) is determined based on this advantage/disadvantage position G (hx, hy). Become. To find the shape of the nasal dorsum, first, the nasal tip position is 1iG (hX).

The position of the pixel G (P
X, Py). That is, Px −hx+5,
It is Py-hy-10. Roughly, as shown in Figure 12,
A region of 20×5 pixels, 20 pixels to the left of G (Px, Py) and 5 pixels below, is defined as the nasal dorsal shape detection region 40. That is, the nasal dorsal shape detection area 40 consists of four points G (Px,
Py ), G (Px , Py +4), G (Px
−19, Pl/) and G (Px −19, Py +4
This is the inside of the rectangle created by >.

Next, G (Px-19, Py +i) and G (Px
.

Py +i) (however, i-0, 1, 2, 3, 4)
The leftmost bright pixel G(P
x −Qi, py +i), and calculate by increasing 1 by 1 from i=Q to i−4. ^The back shape is G
(Px −Qi, Py +i) (i −Q
, 1°2.3.4>.

Step 105 continues once the nasal dorsum shape is determined in this way.
Then, the slope of the straight line connecting the nasal tip and the nasal root is determined using the advantageous and disadvantageous positions and a point representing the nasal dorsum determined from the nasal dorsal shape. Determine the point G (QX, ay) that represents the backbone as follows. The number of pixels of the nose in the back shape detection area 40 is N=ΣQ;

A rectangle having approximately the same number of pixels as the pixel ViN of the nose,
That is, G (Px, Py), G (QX, Py
), G (PX, Py +4), G (qx,
Rectangular G (qx
, Py ), G (QX.

Py+4> Point G(QX) representing the back of the midpoint on the side.

qy). Hit, qx=Px-N15, qx=Py
+2. (qx is the integer closest to px - N15)
The slope k of the straight line connecting the tip of the nose and the root of the nose (the so-called nose base) is k − based on the above advantage/loss position G (hx, hy) and the point G (qx, qy) representing the back of ζ).
It is calculated as (hy-QV)/(hx-QX).

In the following step 106, based on the eye tip position Q (hx, hy) on the pixel data obtained in steps 103 to 105 and the slope k of the straight line connecting the nasal tip and the nasal root, the following formula % formula % The eye position G (mx, my) is calculated using For the values of elevation 1 and elevation 2, images of multiple vehicle drivers may be cropped in advance and the average data thereof may be used.
The relationship between hx, hy) and the 0-spine shape detection area 40 is as shown in FIG.

Once the eye position G (mx, my) on the pixel data is obtained through the processing in steps 102 to 106, the process moves to step 107, where the three-dimensional position of the driver 1's eyes in the vehicle interior is determined. Desired.

In this embodiment, since the image detection unit 6 is fixed, the eye position G (n+x, my) obtained in step 106 is used, and the driver 1 is assumed not to move in the left-right direction. As a result, the three-dimensional position of the eyes can be found. In other words, the left and right sides (Z) of driver 1 shown in FIG.
Assuming that there is no movement in the direction, the three-dimensional position @M of driver 1's eye (left eye) based on the standard point O is expressed as the coordinates (X, Y
, O), the position of the eyes can be recognized. In this process, since the image detection section 6 is fixed, the position G (ox, oy) of the standard point O on the image data is stored in advance, and this standard position Q (ox, oy) is Eye position G (mx,
This can be easily carried out by calculating the deviation of my).

As explained above, the eye position recognition device of this embodiment processes image data obtained by one image detection unit 6 provided diagonally in front of the driver's seat 1, and determines the advantageous and disadvantageous positions of the driver 1. Driver 1
It is configured to recognize the position of the eye on the image data, and then recognize the four-dimensional position of the eye. Therefore, compared to simply determining the eye position from the position of the longevity on the image data, it is possible to recognize the three-dimensional position of the eye with higher accuracy, and it is possible to directly detect the eye position on the image data and calculate the three-dimensional position of the eye. Unlike the case of position recognition, when the driver 1 is wearing glasses or sunglasses, the problem of not being able to recognize the position of the eyes does not occur. Therefore, when the eye position recognition device of this embodiment is used to automatically adjust the angle of the rearview mirror or prevent drowsy driving, it is possible to perform control with higher accuracy. In this embodiment, a MOS type solid-state imaging device is used as the image detection section 6, but other solid-state imaging devices such as COD may also be used. Furthermore, although a non-interlace method is used in this embodiment, an inklace method may also be used.

In addition, in this embodiment, the process executed in steps 102 and 103 in the eye position recognition process shown in FIG. The process executed in step 104 corresponds to V, the process executed in step 105 corresponds to the inclination calculation means Vl, and the process executed in step 105 corresponds to the eye position calculation means Examples include the processing that is executed f7 in steps 106 and 107.

[Effects of the Invention] As described in detail above, the vehicle driver's eye position recognition device of the present invention includes a light emitting means for illuminating the vehicle driver, and an image of the light reflected from the driver by the illumination of the light emitting means. The apparatus includes an image detection means for detecting data as data, and is configured to recognize the position of the vehicle driver's eyes from the slope of a straight line connecting the point of advantage or disadvantage on the image data and the root of the tip of the nose. Therefore, even if the vehicle driver is wearing glasses or sunglasses, the position of the eyes can be recognized well, and the position of the eyes can be recognized with higher accuracy than when simply recognizing the position of the eyes from one singular point position. be able to recognize it.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of the present invention, FIGS. 2 to 14 show an embodiment of the present invention, and FIG. 2 is a perspective view showing the instrument panel of the vehicle of this embodiment and its surroundings. , FIG. 3 is an explanatory diagram of the light emitting unit 5, FIG. 4 is an explanatory diagram of the image detecting unit 6, FIG. 5 is an overall configuration diagram of the eye position recognition device of this embodiment, and FIG. 6 is an explanatory diagram of the eye position recognition device. 7 is an explanatory diagram of the light emission process, and is a time chart showing the relationship between the vertical synchronization signal, the irradiation signal, and the light emission timing of the light emitting section 5. FIG. 8 is a flowchart showing the process, and FIG. 9 is an explanatory diagram showing image data, and FIG. 10 is a data diagram showing a standard nose pattern.
Fig. 11 is an explanatory diagram of the nose position detection process, Fig. 12 is an explanatory diagram of the slope calculation process of the straight line connecting the nasal tip and the nasal root, and Fig. 13 is an explanatory diagram of the process of detecting the position of the nose.
The figure is an explanatory diagram of the eye position calculation process, and FIG. 14 is an explanatory diagram explaining the three-dimensional position of the eyes, in which (a) is a plan view of the driver 1, and (b) is a side view of the driver 1. It is. ■...Light emitting means■...Image detection means■...β cusp position detection means■...Nose dorsal shape detection means■...Inclination calculation means Vl...Eye position calculation means 5... - Light emitting section 6... Image detection section 8...
Image signal processing section 10...image data input section 12...irradiation signal output section 14...CPU

Claims (1)

[Scope of Claims] 1. A light emitting means for illuminating a vehicle driver; and an image detecting means for receiving reflected light from the driver caused by the illumination of the light emitting means and detecting at least the face of the driver as image data. , a nose tip position detection means for detecting the nose tip of the driver in the detected image data; a nose tip position detection means for detecting the nose tip of the driver in the detected image data; a slope calculation means for calculating the slope of a straight line connecting the nose tip and the nasal root based on the nose tip position and the nasal dorsal shape; An eye position recognition device for a vehicle driver, comprising: an eye position calculation means for calculating the position of the vehicle driver. 2. The inclination calculation means calculates the nasal tip and nasal root using a point representing the nasal dorsum obtained from the nasal dorsum shape within a rectangular area on the image data including the nasal dorsum extending a predetermined distance from the nasal tip and the nasal tip position. 2. The vehicle driver's eye position recognition device according to claim 1, which calculates the slope of a straight line connecting the image data. 3. The vehicle driver's eye position recognition device according to claim 1 or 2, wherein one image detection means is provided diagonally in front of the driver's seat. 4. The vehicle driver's eye position recognition device according to any one of claims 1 to 3, wherein the image detection means comprises a two-dimensional solid-state image sensor. 5. The vehicle driver's eye position recognition device according to any one of claims 1 to 4, wherein the light emitting means is configured to emit infrared light.