JPS6177706A - Device for recognizing riding position of vehicle driver - Google Patents

Abstract

PURPOSE:To make it possible to recognize the position of a driver accurately even during daytime, by computing the image data of only reflected light based on the lighted image and the unlighted image of the vehicle driver, which are obtained by controlling a light emitting means and an image detecting means, and recognizing the three-dimensional position. CONSTITUTION:An image detection control means IV controls a light emitting means II and an image detecting means III. The image detecting means III detects a lighted imaged and an unlighted image when the means II emits the light and does not emit the light. Based on the detected lighted image and unlighted image, an image data computing means V computes the image data comprising only the reflected light caused by the emission of the light from the means II. An image processing means IV carries out the processing of the computed image data and recognizes the three-dimensional position of the driver I in a vehicle.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is a vehicle driver riding position recognition device that captures the riding state of a vehicle driver as an image and recognizes the three-dimensional position of the vehicle driver in the cab. In particular, a vehicle driver configured so that the image used to recognize the driver's three-dimensional position is not affected by light from outside the vehicle, such as sunlight entering through the window glass. The present invention relates to a vehicle position recognition device.

[Prior art] With the rapid development of electronic controllers in recent years, research has been underway into new devices aimed at improving vehicle operability, comfort, accident avoidance, etc. Recognizes the riding position, such as the position of the driver's eyes and head, and adjusts the rearview mirror and headrest.

Ideas include automatically adjusting the position and angle of the air outlet or steering wheel, or detecting drowsiness while driving based on periodic changes in the position of the driver's eyes or head and issuing a warning to the driver. It is being The riding position recognition device for recognizing the riding position of the vehicle driver, which is necessary to execute such control, uses an image sensor such as a camera that captures the riding position of the vehicle driver and detects it as an image. The detection unit analyzes the detected image using so-called image processing, detects a predetermined position such as the position of the driver's eyes or head on the image, and detects that part in the vehicle seedling. It has been proposed to recognize the riding position of the driver by incorporating an image processing unit such as a microcomputer that calculates the three-dimensional position of the vehicle.

Furthermore, it is desirable that the image representing the riding condition of the vehicle driver detected by the image detection unit be an image in which the driver is clearly captured, but if the vehicle driver is simply captured using a camera or the like, At night, it's too dark to do m@,
Furthermore, even during the daytime, it is brighter outside the vehicle than inside the vehicle, making it difficult to detect a good image. Therefore, it has been considered to provide a light emitting section that illuminates the driver at the same time when the vehicle driver is imaged. The light emitting section that illuminates the driver is configured to emit infrared light, since irradiation with visible light would cause the driver to feel dazzled, and the image detection section also irradiates the driver with infrared light. It has been considered that an image can be detected using light reflected from a driver.

However, when the vehicle driver is irradiated with infrared light as described above and the reflected light is used to detect the driver's riding status as an image, light from outside the vehicle, such as direct sunlight or reflected light from buildings, is A nighttime image that does not enter the interior of the car can provide a clear image for the driver, but when infrared light emitted from the sun enters the interior of the car during the day, the image is distorted and the driver is There is a problem that the vehicle cannot be captured as a clear image, and in such a case, the driver's riding position cannot be detected satisfactorily.

[Purpose of the Invention] Therefore, the present invention executes image processing in the image processing unit, and the image used when calculating the riding position of the vehicle driver is clearly visible to the vehicle driver regardless of the light from outside the vehicle. It was captured by j! It is an object of the present invention to provide a device for recognizing the riding position of a vehicle driver, which is configured to accurately recognize the riding position of the vehicle driver by controlling the vehicle driver so that the vehicle driver's riding position is controlled to be the same as the vehicle driver's riding position.

[Structure of the Invention] As shown in FIG. 1, the structure of the present invention to achieve the above object includes a light emitting means (2) for illuminating the vehicle driver, and a system for capturing an image of the riding state of the vehicle driver and detecting it as an image signal. and controlling the light emitting means "and the image detecting means (2) so that the image detecting means (2) detects an irradiated image and a non-irradiated image when the light emitting means (2) is irradiated and not irradiated, respectively. The above-mentioned image is detected by the operation of the image detection control means l■ and the image detection control means ■.

an image data calculation means V that calculates image data consisting only of reflected light from the irradiation of the light emitting means (2) based on the irradiation image and non-irradiation image obtained by the detection means (2); and the calculated image data. The gist of the present invention is to provide an apparatus for recognizing the riding position of a vehicle driver, comprising: an image processing means (2) for performing image processing and recognizing the three-dimensional position of the vehicle driver in the vehicle;

[Example] An embodiment of the present invention will be described below with reference to the drawings.

FIG. 2 is a perspective view showing the instrument panel part and its surroundings of a vehicle in which the occupant position recognition device of this embodiment is installed, in which 1 represents the driver, 2 represents the driver's seat, and 3 represents the instrument panel. ing. A light emitting unit 5 that illuminates the upper body of the driver 1 from the left side is located diagonally forward to the left of the driver's seat 2 of the instrument panel 3, that is, in front of the passenger seat side. An image detection unit 6 that can generate a two-dimensional image is provided.

Here, the light emitting unit 5 corresponds to the above-mentioned light emitting means (2), and an infrared strobe that emits infrared light is used so as not to dazzle the driver 1 during irradiation, as shown in FIG.
An infrared light emitter 5a, a lens 5b for broadly irradiating the driver with infrared light emitted by the infrared light emitter 5a, and an infrared filter 5C that transmits infrared light but blocks visible light. , a case 5d for storing the infrared light emitter 5a, lens 5b, and infrared filter 5C and for attaching it to a predetermined position on the instrument panel 3.

Further, the image detection section 6 corresponds to the above-mentioned image detection means m, and as shown in FIG. For example, the focal length for forming an image on the imaging plane is 12.51QIl.
1 lens 6b, a liquid crystal aperture element 6C for adjusting the amount of light, a phototransistor 6d that is used to automatically adjust the aperture of the liquid crystal aperture element 6C and detects the principle of transmission through the liquid crystal aperture element 6C, An MO consisting of a photosensor and a switching circuit that extracts the image formed on the imaging surface through the infrared filter 6a, lens 6b and liquid crystal aperture element 6C as an electrical signal by switching scanning.
A solid-state camera is used that includes an S-shaped solid-state image sensor 6e and a case 6f for storing the above-mentioned parts and attaching them to the instrument panel 3.

Next, the overall configuration of the riding position recognition device of this embodiment will be explained based on the block diagram shown in FIG. 5.

As shown in the figure, in addition to the light emitting unit 5 and the image detecting unit 6, this recognition device processes the image (9) of the driver 1 captured by the image detecting unit 6, and calculates the riding position of the driver 1. In the embodiment, an image processing unit 8 is provided that recognizes the position of the eyes. The image processing section 8 controls the image detection section 6, converts the image signal obtained by the image detection section 6 into a digital signal, and temporarily stores the converted digital signal as image data. an irradiation signal output section 12 that outputs an irradiation signal for causing the light emitting section 5 to emit light; and an irradiation signal output section 12 that outputs an irradiation signal for causing the light emitting section 5 to emit light. A series of eye position recognition processes are executed, such as detecting a predetermined singular point position @ (in this embodiment, an outside position) on the face of driver 1 and recognizing the position of the eyes of driver 1 from this singular point position. A central processing unit (CPU) 14 and a read-only memory (ROM) in which control programs and data for executing eye position recognition processing by the CPU 14 are stored in advance.
16, and a random access memory (RAM) in which data used to execute the Enhan process is temporarily read.
) 18, a path line 20 that connects each of the above sections and serves as a path for image signals and control signals, and a power supply circuit 22 that supplies power to each of the above sections. The image input section 10 also includes a synchronization signal generation circuit 10a that generates a vertical synchronization signal and a horizontal synchronization signal to the image detection section 6, and an image signal obtained by the image detection section 6, for example, at 256 pixels horizontally per frame. , an A/D conversion circuit 10b that converts into image data consisting of digital signals representing the brightness of 240 vertical pixels;
It is comprised of an image data storage circuit 10C that can store the converted image data for the past two frames.

Next, the operation of the eye position recognition process executed by the image processing section 8 will be described in detail with reference to the flowchart shown in FIG. This process is executed when a request for recognizing the position of the driver's 1 eyes is inputted by a signal of a certain period or a signal from a switch. The position of the eyes will be recognized as the riding position.

As shown in the figure, when the process is started, step 101 is first executed, and in the image detection unit 6, an irradiation image as shown in FIG. , an irradiation signal is output to the light emitting unit 5, and the irradiation signal is outputted to the light emitting unit 5 so that two types of images are detected: a non-irradiation image as shown in FIG. An image data reading process is executed in which the two types of detected images are stored in the image input unit 10 as irradiation image data G1 and non-irradiation image data G2, respectively, each consisting of a digital signal.

Here, the image input section 10 includes a synchronization signal generation circuit 10a,
It is equipped with an A/D conversion circuit 10b and an image data storage circuit 10c, and controls the image detection section 6 according to the vertical synchronization signal and horizontal synchronization signal output from the synchronization signal generation circuit 10a, and outputs the image data from the image detection section 6. The image signals that are sequentially taken in and converted into digital signals via the A/D conversion circuit 10b can be stored in the storage circuit 10G as irradiation image data G1 or non-irradiation image data G2. The data reading process can be easily executed by simply operating the image input section 10 and causing the light emitting section 5 to emit light in synchronization with the vertical synchronization signal output from the synchronization signal generation circuit 10a. In other words, as shown in Figure 9, first! ll! After a predetermined period of time Δt has elapsed since the rising time to of the Hyakudoumei signal, the irradiation signal S is output to the light emitting unit 5 to cause the light emitting unit 5 to emit light in coincidence with the vertical retrace time Δt1. 7 is read as irradiation image data G1, and in the next frame, a non-irradiation image as shown in FIG. 8 obtained without emitting light from the light emitting unit 5 is read as non-irradiation image data G2. It will be executed.

7 and 8 are image diagrams in which the light incident from outside the vehicle is strong and the scenery outside the vehicle is also imaged at the same time as the driver 1 is imaged, which is represented by the density of diagonal lines. In the irradiation image shown in the figure, even when the light emitting unit 5 emits light and infrared light is irradiated to the driver 1, the incident light from outside the vehicle is stronger than the reflected light from the face of the driver 1. The non-illuminated image shown in FIG. 8 is an image made only of incident light from outside the vehicle, and shows that the driver 1 is completely in the shadow. Further, image data G1 which is A/D converted and stored in the image input section 1o.
(or G2) is the image obtained by the image detection unit 6 as described above, horizontally 256,! It represents the brightness of pixels divided into 111240, and the brightness G1<x, y) is stored for each pixel as shown in FIG.

In this way, in step 101, the light emitting section 5, the image detecting section 6
and the image input unit 1Q are controlled, and when the two types of image data G1 and G2 are read in the image input unit 10, the subsequent step 102 is executed, and the two types of image data G1
and G2, image data G1 consisting only of reflected light from the light emitted by the light emitting unit 5, that is, an image representing a clearly captured image of the driver 1 inside the vehicle regardless of the scenery outside the vehicle, as shown in FIG. Processing to calculate data G is performed.

Note that the process of step 102 is the same as that of step 10 above.
This can be easily carried out by calculating the difference between the irradiation image data G1 and the non-irradiation image data G2 stored in the image input unit 10 in the step 1 for each pixel.

When the image data G is calculated in step 102, the next step 103 is executed, and a predetermined area in the image data G, that is, 30 areas shown in FIG. Search for pixels that exist on the leftmost side of the area and have a brightness above a certain level,
A bright spot search process is performed in which the position is stored. This process is a process for limiting the range in which the nose of the driver 1 is detected when detecting the position of the nose of the driver 1 in the next step 104, and the position of the determined pixel is the starting point for searching for the nose. It is said that In this bright spot search process, starting from the upper left pixel of the nose-existing relative 3o in FIG. It can be executed by memorizing it.

When the bright spot search is executed in this way and the position of the pixel that exists on the leftmost side of the nose position and has a brightness above a certain level is determined, in the subsequent step 104 as described above,
Processing is performed to search for the nose using this position as a base point and determine the position of the nose.
The degree of correlation between the standard pattern of the nose as shown in the figure and the image data is examined, and the position of the image where the correlation is maximum is regarded as the position where the nose exists, and the position of the nose is determined.

In other words, centering on the pixel position G (x, y) determined in step 103 above, as shown in FIG. The center position in the standard pattern of 碍 shown in @!I
The correlation value between the image data and the standard pattern is calculated as l (O, O), and the pixel position where the value is the largest is recognized as the position of gain or loss (so-called tip of the nose) G < hx, hy). . Here, the center position @m (0,0) in the standard nose pattern shown in Fig. 12 is preset as a portion corresponding to gains and losses, and the correlation value can be obtained from the following formula. . As for the values of i and J above, the standard nose pattern is configured as data with a spread of 16 above, 7 below, 1 on the left, and 6 on the right with respect to the center m (0, O). Therefore, it is sufficient to use an integer value between (-1) and (+6) for i and between (-16> and (+7)) for j.

Also, the correlation value is the center position In in the standard pattern of the nose.
It is calculated by applying a total of 121 pixels to (0, 0), 5 pixels each on the top, bottom, left and right, centering on pixel G (X, y) on the image data as shown in Figure 13. Therefore, the values of XSY in the above formula are, respectively,
X is an integer value between ×±5 and Y is an integer value between y±5, and in the end, a maximum of 121 types of correlation values are produced by the combination of X and Y, and the maximum value is the pixel G (X, Y)
is detected as the nose position G (hx, hy).

Once the nose position G (hX, hy) has been determined in this way, the process moves to the subsequent step 105, where the eye positions are determined based on the nose position G (hx, hy). To find the position of the eyes, first select 12 pixels to the right and 13 pixels above the nose position G (hx, hy).
Pixel position G (hx+12, hy-13
＞, and find the 14th point set in advance near this position.
The degree of correlation between the eye standard pattern shown in the figure and the image data is examined to determine the eye position Q (mx, mV). Note that this determination method can be carried out in the same manner as the nose position determination in step 104 above, so the explanation will be omitted.

In the following step 106, the eye position G (mx,
my), a process is executed to determine the three-dimensional position of the driver's 1 eyes in the vehicle interior, assuming that the driver 1 does not move in the left-right direction. In other words, assuming that the driver 1 does not move in the left and right <2> directions shown in FIG. )'s three-dimensional position M as the coordinates (X, Y, O)
The position of the eyes is recognized by determining the position of the standard point P on the image data, since the image detection unit 5 is fixed.
(px, py) is stored in advance, and this reference position G
Eye position G(mx, my) relative to (px, py)
This can be easily carried out by calculating the deviation of .

As explained above, in the vehicle driver's riding position 2 recognition device of this embodiment, the light emitting unit 5 and the image detecting unit 6 are provided on the passenger seat side instrument panel diagonally in front of the driver 1. The image obtained by the image detection unit 6 when the light emitting unit 5 is illuminated is set as irradiation image data G1, and the image obtained by the image detection unit 6 without irradiation of the light emitting unit 5 is set as non-irradiation image data G2. After that, these two types of image data G
The image data G used for recognizing the riding position (eye position) of the driver 1 is calculated from the difference between G1 and G2. Therefore, even in a case where the light incident from outside the vehicle is detected by the image detection unit 6 and a colored image of the outside of the vehicle is captured, by taking the difference between the irradiation image data G1 and the non-irradiation image data Gz, The part imaged by light from outside the vehicle can be removed, and the driver 1 can actually
When executing the process of recognizing the riding position of the driver 1, the process can be executed using image data G representing a clearly captured image of the driver 1 without being affected by light from outside the vehicle, and the process of recognizing the riding position (eyes) Become able to accurately recognize the location of In addition, in this embodiment, instead of directly determining the position of the driver's eyes from the image data G, the position of the nose that protrudes from the face, which is the part that is easiest to detect on the image data, is first determined as the singular point position. After the detection, the eye position is detected and its three-dimensional position is calculated, and furthermore, when detecting the ^ position and the eye position, it is configured to be executed using each standard pattern. This makes it possible to more accurately recognize the riding position without erroneously recognizing the three-dimensional position of the eyes.

Here, in the above embodiment, the irradiation image and the non-irradiation image are temporarily stored as the irradiation image data G1 and the non-irradiation image data G2, respectively, and then the image data G is calculated. After that, read the non-irradiation image data, export the image data G at the time of input,
The image data G may be stored, and in this case, it is not necessary to store the non-irradiation image data G2. Alternatively, the non-irradiation image data G2 may be read first, and then the irradiation image data G1 may be read.

Furthermore, in the above embodiment, the position of the eyes of the driver 1 (more precisely, the position of the left eye) is recognized as the riding position of the vehicle driver. The three-dimensional position of the driver 1's nose may be recognized as the riding position, and the processing after recognizing the riding position, such as automatically adjusting the rearview mirror or automatically adjusting the steering angle, may be performed. The configuration may be such that the most appropriate riding position is recognized according to each process such as. Incidentally, when the position of the eyes of the driver 1 is recognized as in the above embodiment, it can be used to automatically adjust the angle of the rearview mirror, detect drowsy driving, etc.

Further, in the above embodiment, the light emitting means (2) includes a light emitting section 5, the image detecting means (2) includes an image detecting section 6, the image detection control means (2), and the image data calculating means v1 and the image processing means Vt include an image processing section. In addition to the image detecting section 6 consisting of a MOS type solid-state image carrier as in the above embodiment, the image detecting means (2) may also be a carrier used in a so-called solid-state camera such as a COD. Any device may be used as long as it consists of an image element, and it is also possible to use an image pickup tube.

Further, in the above embodiment, the number of image detection units 6 is one, and the driver's riding position is recognized using image data captured from one direction, and the driver's movement in the left and right direction is Although this has not been taken into account, if two image detection units are installed at different positions and configured to recognize the driver's boarding position using the so-called triangular colorimetry principle, it will be possible to prevent the driver from moving in the left or right direction. The three-dimensional position of the driver can be detected more accurately.

[Effects of the Invention] As described in detail above, in the device for recognizing the riding position of a vehicle driver of the present invention, the irradiated image of the vehicle driver obtained by controlling the light emitting means and the image detecting means, the irradiated image and the non-irradiated image. Based on the image, image data consisting only of reflected light emitted by the light emitting means is calculated, and the three-dimensional position of the vehicle driver is recognized from the calculated image data. Therefore, even if the image of the vehicle driver obtained by the image detection means is affected by sunlight, etc. that enters through the vehicle window glass, and the vehicle driver is not clearly displayed, the actual The image data used to recognize the three-dimensional position of the vehicle driver is an image with light from outside the vehicle removed, that is, an image in which the vehicle driver is clearly captured (image data representing an elephant). This makes it possible to accurately recognize the vehicle driver's riding position even during the day.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of the present invention, FIGS. 2 to 15 show an embodiment of the present invention, and FIG. 2 is a perspective view showing the instrument panel of a vehicle and its surroundings according to the present embodiment. , FIG. 3 is an explanatory diagram of the light emitting section 5, FIG. 4 is an explanatory diagram of the image detecting section 6, FIG. 5 is an overall configuration diagram of the riding position recognition device of this embodiment, and FIG. 6 is an explanatory diagram of the image processing section 8 A flowchart showing the operation, Figure 7 is an image diagram showing the irradiation image, and Figure 8 is a flowchart showing the operation.
The figure is an image diagram showing a non-irradiation image, and Figure 9 is image data G.
1, a time chart explaining the G2 reading process, and FIG. 10 shows the irradiation image data G stored in the image input unit 10.
An explanatory diagram showing 1, FIG. 11 is an image diagram of image data G,
Figure 12 is a data diagram representing the standard pattern of the nose, Figure 13
FIG. 14 is an explanatory diagram of the nose position detection process, FIG. 14 is a data diagram showing the standard eye pattern, and FIG. Plan view,
(b) is a side view of the driver 1. ■... Light emitting means ■... Image detecting means ■... Image detection control means V... Image data calculating means ■... Image processing means 5... Light emitting section 6... Image detecting section 8 ...Image processing section 10...Image input section 14...CPU

Claims (1)

[Scope of Claims] 1. A light emitting means for illuminating a vehicle driver; an image detecting means for capturing an image of the riding state of the vehicle driver and detecting it as an image signal; and the image detecting means illuminates the light emitting means. an image detection control means for controlling the light emitting means and the image detection means so that an irradiation image and a non-irradiation image are detected at the time of irradiation and at the time of non-irradiation, respectively; an image data calculation means for calculating image data consisting only of reflected light from the irradiation of the light emitting means based on the obtained irradiation image and non-irradiation image; and image processing of the calculated image data; 3 of the above vehicle driver in the vehicle
An apparatus for recognizing a riding position of a vehicle driver, comprising: an image processing means for recognizing a dimensional position; 2. Claim 1, wherein the light emitting means emits infrared light and the image detection means is configured to receive infrared light.
A device for recognizing the riding position of a vehicle driver as described in 2. 3. Recognition of the riding position of a vehicle driver according to claim 1 or 2, wherein the image detection means is arranged on the passenger seat side of the vehicle so as to detect the riding state of the vehicle driver as a side image. Device. 4. An apparatus for recognizing the riding position of a vehicle driver according to any one of claims 1 to 3, wherein the image detection means comprises a two-dimensional solid-state image sensor. 5. The device for recognizing the riding position of a vehicle driver according to claims 1 to 4, wherein the three-dimensional position of the vehicle driver recognized by the image processing means is the position of the vehicle driver's eyes.