JPS6168503A - Vehicle operator position recognition equipment - Google Patents

Abstract

PURPOSE:To enable the recognition of a specified part of an operator accurately, by detecting a reflected light room the operator with the irradiation from a light emitting means as image data to recognize a specified part once a back ground data is attenuated by a background processing means. CONSTITUTION:In an apparatus for detecting a specified part, for example, the positions of the eyes and nose of a vehicle operator I, a light emitting means II for irradiating the operator I and an image detection means III which detects the operator I as image data receiving a reflected light therefrom I with the irradiation of the light emitting means are provided. Image data obtained from the image detection means III are inputted into a background processing means IV to attenuate background data of a specified part based on data previously memorized. Then, the image data background processed is inputted into a position recognition means V to recognize the position of the specified part of the operator I.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a vehicle driver position recognition device that captures the riding state of a vehicle driver as an image and recognizes the vehicle driver's position in the vehicle interior. The present invention relates to a vehicle driving position recognition device configured to accurately recognize the position of a predetermined part of a vehicle driver.

[Conventional technology] With the rapid development of electronic equipment in recent years, new devices have been developed to improve vehicle operability, comfort, accident avoidance, etc.
Research is underway, one of which is to recognize the riding position of a vehicle driver, such as the position of the driver's eyes and head.
Automatically adjust the rotation and angle of rearview mirrors, headrests, air conditioning air outlets, steering wheels, etc.
It is being considered that drowsy driving can be detected based on periodic changes in the position of the driver's eyes or head, and a warning can be issued to the driver. A riding position recognition device for recognizing the riding position of the vehicle driver, which is necessary to execute such control, is a device that images the riding state of the vehicle driver,
An image detection unit consisting of a camera or the like that detects an image, and the detected image is analyzed using so-called image processing.
An image processing unit consisting of a microcomputer etc. detects a predetermined position such as the position of the driver's eyes or head on the image and calculates the position of that part in the vehicle interior. The idea is to recognize the driver's location.

[Problems to be solved by the invention] Automatic adjustment of the positions and angles of the above-mentioned rearview mirrors, headrests, air-conditioned air outlets, steering wheels, etc., and periodic changes in the position of the driver's eyes and head. In order to detect drowsy driving based on changes in driving and issue a warning to the driver, it is necessary to accurately recognize predetermined parts of the driver, such as the positions of the eyes and nose.

SUMMARY OF THE INVENTION An object of the present invention is to provide a vehicle driver position recognition device that can accurately recognize a predetermined part of the driver.

Means for Solving Problem L] As shown in FIG. Upon receiving the reflected light, 1iI
(ii) an image detection means (2) for detecting a driver as image data; and (2) a background processing means (2) for attenuating background data of a predetermined part in the vicinity of a predetermined part of the driver who is applied to the detected image data; The gist of the present invention is a vehicle driver sentiment recognition device comprising: a position recognition means V for recognizing the position of the predetermined part of the driver (2) in background-processed image data;

[Function] -5 - In the present invention, in order to obtain a clear image in the image detection means, a light emitting means is provided as a light source other than natural light, and the light emitting means illuminates the vehicle driver to brighten it. In this case, it is preferable to irradiate infrared light because the driver will not feel dazzled by the light emitting means.

The image detection means receives reflected light from the driver due to irradiation from the light emitting means, and detects information necessary for position recognition as image data of the driver. In this case, it is preferable to arrange the image detection means on the passenger seat side so as to detect a side image of the driver so that characteristic parts of the face such as the nose and chin can be easily recognized. Furthermore, if a two-dimensional solid-state image sensor is used as the image detection means,
It is preferable because it does not require space and has excellent impact resistance.

The background processing means is used as pre-processing for the position recognition means for the predetermined region, and clarifies the shape of the predetermined region on the image data by reducing the background of the predetermined region in the vicinity of the predetermined region.

The position recognition means recognizes the position of the predetermined region based on the shape of the predetermined region clarified by the background processing means. Further, during this recognition, it is preferable to recognize the position by comparing image data near the predetermined region with a standard pattern of the predetermined region, since recognition errors will be further reduced.

[Example] A first embodiment of the present invention will be described below with reference to the drawings.

FIG. 2 is a perspective view showing the instrument panel part and its surroundings of a vehicle in which the occupant position recognition device of this embodiment is installed, in which 1 represents the driver, 2 represents the driver's seat, and 3 represents the instrument panel. ing. Then, diagonally forward to the left of the driver's seat 2 of the instrument panel 1 to panel 3, that is, in front of the passenger seat side, there is a light emitting unit 5 that illuminates the upper body of the driver 1 from the left, and a light emitting unit 5 that illuminates the upper body of the driver 1 from the left. An image detection unit 6 is provided that can generate a two-dimensional image from the image.

Here, the light emitting unit 5 corresponds to the above-mentioned light emitting means (3), and is an infrared robot that emits infrared light so as not to dazzle the driver 1 during irradiation, as shown in FIG. As shown in FIG. Outer filter 5C, these infrared emitters 5a, lens 5b and infrared filter 5C
and a case 5d for storing and attaching it to a predetermined position on the instrument panel 3.

Further, the image detection section 6 corresponds to the above-mentioned image detection means (2), and as shown in FIG. A lens 6b with a focal length of, for example, 12.5 mm for forming an image on the image plane, a liquid crystal aperture element 6C for adjusting the front projection, and a liquid crystal aperture element 6C used to automatically adjust the aperture of the liquid crystal aperture element 6C. Photo 1 detects the amount of light transmitted through the aperture element 6C.
~ Consists of a transistor 6d, a photodiode array, and a switching scan, which extracts the image on the ML image plane that is formed after passing through the infrared filter 6a, lens 6b, and liquid crystal aperture element 6C as an electrical signal by switching scan. A solid-state camera is used, which is composed of a MOS type solid-state image element 6e and a case 6f for storing the above-mentioned parts and taking it to the instrument panel 3.

Next, the overall configuration of the riding position recognition device of this embodiment will be explained based on the block diagram shown in FIG.

As shown in the figure (in addition to the light emitting unit 5 and image detecting unit 6), this recognition device processes the image of the driver 1 captured by the image detecting unit 6, and processes the driver's position (in this embodiment). is equipped with an image processing unit 8 that recognizes the position of the nose).The image processing unit 8 controls the image detection unit 6, and
An image input section 10 that converts the image signal obtained by the image detection section 6 into a digital signal and temporarily stores the converted digital signal as image data, and an image input section 10 for causing the light emitting section 5 to emit light. The irradiation signal output unit 12 outputs an irradiation signal, and the image input unit 10 A/D-converts and stores the image data, and performs background processing on the background near a predetermined part of the face of the driver 1. ! A central processing unit (CPU) 14 executes a series of position recognition processes such as recognizing the position of the nose of the driver 1 from image data that has been processed, and a control program and a control program for executing the position recognition process in the CPU 14. Read-only memory (ROM) 16 in which data is stored in advance
A random access memory (RAM) 1 in which data used for executing arithmetic processing is temporarily read and written.
8, a path line 20 that connects the above-mentioned parts and serves as a path for image signals and control signals, and a power supply circuit 22 that supplies power to the above-mentioned parts. In addition, the image input section 10
The synchronizing signal generating circuit 10a which generates a vertical do-bright signal and a horizontal do-bright signal to the image detecting part 6 outputs the image signal obtained by the image detecting part 6, for example, by horizontally 25 times per frame.
6. An A/D conversion circuit 10b that converts image data consisting of digital signals representing the brightness of 240 vertical pixels, and an image data storage circuit 100 that can store the converted image data for the past two frames. It is composed of.

Next, the operation of the position recognition process executed by the image processing section 8 will be explained in detail along the line 70 shown in FIG. Note that this process is executed when a request for recognizing the position of the driver 1 is inputted by a signal of a certain period or a signal from a switch, etc., and as described above, in this embodiment, the position of the driver 1 is determined as the position of the driver 1. There is a roar as the position of the nose is recognized.

As shown in the figure, when the process starts, first step 101
is executed, and the image detection unit 6 outputs an irradiation signal to the light emitting unit 5 so that an image is detected when the light emitting unit 5 emits light, and also outputs the detected image to the image (qξ input unit 1
ii! is stored as image data G consisting of digital signals at 0! ii Image data reading processing is executed.

Here, the image input section 10 includes a synchronization signal generation circuit 10a,
It is equipped with an A/D conversion circuit 10b and an image data storage circuit 10C, and controls the image detection section 6 according to the vertical synchronization signal and horizontal synchronization signal output from the synchronization signal generation circuit 10a, and outputs from the image detection section 6. Image signals are sequentially captured and converted into digital signals via the Δ/D conversion circuit 10b, and the image signals can be stored as image data G in the storage circuit 10C. 10 and causes the light emitting section 5 to emit light in synchronization with the vertical dosing signal output from the synchronizing signal generating circuit 10a. That is, as shown in FIG. 7, first, after a predetermined time Δt has elapsed from the rise time to of the vertical synchronization signal, the irradiation signal S is output to the light emitting unit 5, thereby causing the light emitting unit 5 to emit light in coincidence with the vertical retrace time Δt1. The image generated by the light emission is read as image data G.

Note that the image data G that is Δ/D converted and stored in the -F old image input section 10 is converted into 1 by the image detection section 6 as described above.
It represents the brightness of pixels that divide the q image into 256 horizontal and w 240 pixels, and the brightness G (X, V>) is stored for each pixel as shown in FIG.

In this way, in step 101, the light emitting section 5, the image detecting section 6
When the image input unit 10 is braked and the image data G is read in the image input unit 10, the subsequent step 102 is executed, and a predetermined area set in advance in the image data G, that is, 30 areas shown in FIG. A bright spot search process is performed in which a pixel having a constant brightness of less than or equal to Bell is searched for in the leftmost region of the nose, and its position is stored. This process is used to perform background processing when processing the back surface near the nose of the driver 1 in the next step 103. This is a process for limiting the range 31, and the determined pixel position is the processing range 31 of background processing.
It is used as the base point for determining.

In this bright spot search process, starting from the upper left pixel in the nose existing area 30 in FIG. It can be executed by memorizing it.

The bright spot search is executed in this way, and the position G(
Once G (Xo, Vo) is obtained, in step 103, which follows as described above, the background processing shown in FIG. The processing range 31 is determined and background processing is performed. Each time the brightness of each pixel is checked, the brightness of the checked pixel is made the same as the darkest pixel from the starting point to that pixel.

FIG. 10 shows an enlarged view of the processing range 31.

The processing range 31 is a point G(xl, Vl>, G(Xt *V
2), G (X 2. Vl) and G (Xl, V2) 4
It is a rectangle surrounded by points. In addition, in FIG. 10, the background omitted in FIG. 9 is represented by the density of diagonal lines.

The background processing starts with the straight line segment with the driver's side G(Xl, V+) as the starting point and G(Xl, Vl) as the ending point, from the starting point G(Xl, 1/1) to the ending point G(Xl, V+). While searching sequentially toward , the brightness of the search point is set to the darkest point from the starting point to the search point. Below, G(Xl, V++1)
From G (Xl.

From G(Xl, V2) to G(Xl
, Vz), the same process is carried out while shifting by one pixel to obtain 31' in FIG. FIG. 10(b) is a diagram showing the brightness distribution of the linear valves from a to b in FIG. 10(a). By performing the background processing described above, as shown in FIG. 211(b), from b'
It gradually becomes darker towards the end of the nose, and there is no brighter area on the left side of the back of the nose. In addition, FIG. 11 is a 1q diagram obtained by processing FIG. 10 as a background, and a' and b- in FIG. 11 correspond to a and b in FIG.
1, G'(X+,'V1) is G(Xt.

V+), there is a one-to-one correspondence.

When the contradiction process is executed in this way, in step 104, which follows as described above, the process of searching for the nose using G(Xo, Vo) as a base point and determining the position of the nose is performed. In this method, the degree of correlation between the preset nose standard pattern shown in Figure 12 and the image data is checked in the vicinity of the pixel found by the bright spot search, and the position of the image where the correlation is maximum is determined. This is assumed to be the position where the nose exists, and determines the position of Kotobuki.

That is, as shown in FIG. 13, centering on the pixel position G (Xo, Vo) determined in step 102 above,
5 pixels each on the left and right, a total of 121 pixels, each on the 12th
1 tril at the center of the standard pattern of the shoe shown in the figure
(0, 0>), the correlation value between the image data and the standard pattern is calculated, and the pixel position where the value is the largest is recognized as the nose position G (hx, hy).Here, the 12th Center position m in the standard pattern of Kotobuki in the figure
(0,0) is set in advance as a portion corresponding to the nose, and the correlation value can be obtained from the following formula. And the above i and j (“1
, since the standard pattern of the nose is configured as data with a spread of 16 above, 7 below, 1 on the left, and 6 on the right with respect to the center m (0, O), 1)
to (+6), and an integer value between (-16) and (+7) may be used for j.

Also, the correlation value is the center position m in the standard pattern of the nose.
It is calculated by applying a total of 121iii1 pixels to (0, O), 5 pixels each on the top, bottom, left and right of the image data shown in Figure 13, centering on pixel G (X, V). , In the above equation, the values of X and Y are integer values between x±5 and Y, respectively.In the end, a maximum of 121 types of correlation values are calculated by the combination of X and Y, and the Pixel G (X,
Y) is detected as the nose position G (hx, hy).

As explained above, in the vehicle driver position recognition device of this embodiment, when recognizing a predetermined region, the predetermined region is clarified in order to process the background in the vicinity of the predetermined region; Since the vehicle is configured to be executed using a standard pattern, the riding position can be recognized more accurately without erroneously recognizing the position of a predetermined part.

Next, a second embodiment of the present invention will be described with reference to the drawings.

In this embodiment, the position of a predetermined part of the vehicle driver 1 other than the nose (in this embodiment, the chin) is recognized based on the 4i1 position of the nose of the vehicle driver 1 recognized in the first embodiment. , Furthermore, the position of the eyes is recognized from the two recognized predetermined parts.

The operation of the position recognition process executed in this embodiment will be described in detail with reference to the flowchart shown in FIG. Note that step 201 and step 202 of this embodiment
, step 203 and step 204 are steps 101, 102 and 10 in the first embodiment.
3 and step 104, so the explanation will be omitted.

By sequentially executing steps 201, 202, 203 and 204, the position of the nose G is determined.
Once (hx, by) is determined, the following step 20
5 and step 206, the position of the chin is determined based on this nose position Q (hx, hy). When processing the background near the chin in step 205, first, the pixel position IG (hx+10
．． tvy-35>, and based on this point, find the blue processing range 32 of the chin in Fig. Process the background.

However, since the shape of the jaw is different from that of the nose, background processing is performed by changing the slope of the straight line.

That is, the processing range 32 in FIG. 15 is treated as a set of parallel straight line segments having a predetermined slope from the upper right to the lower left, and each straight line segment is processed. For each straight line segment, background processing is performed in the same manner as in step 203 above, with the upper right end as the starting point and the lower left end as the ending point. Next, the process moves to step 206 and G (hx+10. hy+35
), the degree of correlation between the image data and a standard jaw pattern as shown in FIG. Note that this determination method can be performed in the same manner as step 204 in determining the nose position in step 104 in the first embodiment, so a description thereof will be omitted. Also, G(hx+1Q, hy+
Point 35) was obtained experimentally from multiple vehicle drivers.

In the following step 207, the nose position G (hx, hy) and the chin position G (ax, ay) on the pixel data obtained in the above steps 204 and 206 are determined.
), G (II
IcX, mcy), and calculate this position Q(mcX
, may) The 17th position is preset in the vicinity of the position.
The correlation between the eye standard pattern shown in the figure and the image data is determined to determine the eye position G (mx, my). Note that this determination method may be performed in the same manner as the nose position determination in step 104 in the first embodiment, and thus the description thereof will be omitted.

In the following step 208, the eye position G (mx,
my), a process is executed to calculate the three-dimensional appearance of the driver's 1 eyes in the vehicle interior, assuming that the driver 1 does not move in the left-right direction. In other words, = 20 − Assuming that the driver 1 does not move in the left/right (Z) direction shown in FIG. Considering the coordinate system where
X, Y, O), the eye position is mtM
However, this process involves the image detection section 5
is fixed, the position G (pX,' IIV) of the standard point P on the image data is stored in advance, and the eye position G (m
This can be easily carried out by calculating the deviation of x, my).

As explained above, in the vehicle driver position recognition device of this embodiment, when recognizing a predetermined region, the predetermined region is clarified in order to perform background processing in the vicinity of the predetermined region, and furthermore, the predetermined region is Since it is configured to be executed using a standard pattern, the position of the predetermined part will not be erroneously recognized, and the riding position can be recognized more accurately.

Furthermore, in this embodiment, instead of directly determining the position of the driver's eyes from the image data, we first recognize the position of the nose and the position of the chin, which are parts that are easy to detect on the image data, which protrude from the face. Then, the position of the eyes is recognized from the correlation between longevity, the chin, and the eyes, so the riding position (the position of the left eye) can be easily recognized.

Further, the eye position recognition device in step 207 of this embodiment can be used as follows.

Based on the nose position G (hx, hy) and the chin position 〇 (ax, ay>) on the pixel data obtained in steps 204 and 206, the following formula (where l), 9.1,
2 and 3 are constants) to determine the eye position G (n+x,
my> is calculated. For the values of Standing 1, Standing 2, and Standing 3, it is sufficient to take images of multiple vehicle drivers in advance and use their average data, and this required eye position G (II
IX, n+y), the relationship between the nose position G (hx, by) and the chin position Q (ax, ay) is as shown in FIG.

By doing so, the position of the eyes can be correctly recognized even when the driver 1 is wearing glasses or the like.

Furthermore, in this embodiment, the position of the eyes of the driver 1 (more precisely, the position of the left eye) is recognized as the riding position of the vehicle driver, but for example, the position of the nose of the driver 1 is The three-dimensional position may be recognized as the position, and the process after recognizing the position of the vehicle driver, such as automatically adjusting the rearview mirror or automatically adjusting the steering angle, may be performed. It may be configured to recognize the most appropriate @weight position. Incidentally, when the position of the eyes of the driver 1 is recognized as in this embodiment, it can be used to automatically adjust the angle of the rearview mirror, detect drowsy driving, etc.

In the background processing means of the first and second embodiments, the straight line segments are parallel to the horizontal direction on the image data, but if necessary, they may be radial with the upper right as the origin. That is, the starting point and the slope may be set so that the starting point of the necessary straight line segment is on the driver's predetermined region so that the predetermined region of the driver remains bright after image processing.

Furthermore, in the first and second embodiments, the light emitting means (2) described above includes a light emitting section 5, the image detecting means (2) includes an image detecting section 6, and the dorsal side processing means (2) v and the position recognition means V each include a light emitting section 5. The image processing section 8 corresponds to each of the above, but the image detection means (2) may include, in addition to the image detection section 6 consisting of a MOS type solid-state image carrier as in the above embodiment, a so-called solid-state camera such as a COD. It is also possible to use an image pickup tube as long as it consists of an image pickup element used in

In addition, in the first embodiment and the second embodiment, the number of image detection units 6 is one, and the riding position of the driver is recognized using image data that captures the driver from one direction.
Although the driver's movement in the left and right directions is not taken into consideration, if two image detection units are installed at different positions and configured to recognize the driver's riding position using the so-called triangulation principle, driving becomes easier. The driver's movement in the left and right directions can also be detected, making it possible to more accurately recognize the three-dimensional position of the driver.

In the first and second embodiments, an infrared strobe is used as the light emitting means (2), but continuous light such as an incandescent bulb may also be used.

Furthermore, in the first and second embodiments, in the bright spot search processing, the search is sequentially performed in the vertical direction starting from the upper left corner, but it is also possible to assign a search ranking to all pixels in the nose region and to determine the ranking. You may search according to the following.

Further, in the first and second embodiments, the nose region is rectangular, but it may be circular or triangular, for example. In this case, a more efficient search can be achieved by combining the above-described method of ranking pixels in a search order.

In addition, instead of directly performing background processing on the image data obtained by the image detection means (■) to recognize the position, if the position is recognized by performing background processing on the image data that has been subjected to etch detection processing and horizontal differential processing, Since the characteristics of the image data become clearer, more accurate position recognition is possible.

[Effects of the Invention] As described above in detail, in the vehicle driver's sentiment recognition device of the present invention, background processing is performed on data near a predetermined portion of image data obtained by the image detection means, thereby eliminating the influence of rear drivers. The vehicle operator is configured to recognize the location of the vehicle operator after decreasing Therefore, it is possible to accurately recognize specific parts of the driver, and automatically adjust the positions and angles of rearview mirrors, headrests, air-conditioned air outlets, steering wheels, etc., as well as the position of the driver's eyes and head. This is useful for detecting drowsy driving based on periodic changes in the number of drivers and providing warnings to the driver.

[Brief explanation of the drawing]

Fig. 1 is a block diagram showing the configuration of the present invention, Figs. 2 to 19 show embodiments of the present invention, and Fig. 2 shows an instrument panel of a vehicle and its surroundings to which the embodiment of the present invention is applied. FIG. 3 is an explanatory diagram of the light emitting unit 5,
FIG. 4 is an explanatory diagram of the image detection section 6, FIG. 5 is the overall configuration of the vehicle position recognition device, FIG. 6 is a flowchart showing the operation of the image processing section 1 8 of the first embodiment, and FIG. FIG. 8 is an explanatory diagram showing the irradiation image data G stored in the image input unit 10; FIG. 9 is an image diagram of the image data G; FIG. 10 (A) , (B) and Figures 11 (A) and (B) are explanatory diagrams of blue stone processing, and Figure 12 is a data diagram representing the standard pattern of the nose.
Fig. 13 is an explanatory diagram of the nose position detection process, and Fig. 14 is an explanatory diagram of the nose position detection process.
A flowchart showing the operation of the image processing unit 8 of the embodiment,
Fig. 15 is an image diagram of image data G, Fig. 16 is a chin mark Q
17 is a data diagram representing a standard eye pattern; FIG. 18 is an explanatory diagram illustrating the three-dimensional position of the eyes; (a) is a plan view of the driver 1;
(B) is a side view of the driver 1, and FIG. 19 is an explanatory diagram of the process of calculating the position of the eyes. ■...Light emitting means■...Image detection means■...Background processing means■...Position recognition means 5...Light emitting section 6...Image detection section 8...Image processing section 10...・Image input unit 14...CPU

Claims (1)

[Scope of Claims] 1. A light emitting means for illuminating a vehicle driver; an image detecting means for detecting the driver as image data by receiving reflected light from the driver caused by the illumination of the light emitting means; and an image detection means for detecting the driver as image data. Background processing means for attenuating background data of a predetermined part of the driver in the vicinity of the predetermined part of the driver in the image data; and position recognition means for recognizing the position of the predetermined part of the driver in the background-processed image data. A vehicle driver position recognition device comprising: 2. The background processing means increases the brightness of pixels on a plurality of straight line segments corresponding to all pixels within a processing range near a predetermined part, one pixel at a time from the starting point to the ending point, starting from the driver's side of each line segment. 2. The vehicle driver position recognition device according to claim 1, wherein the device is configured to perform data processing to make the brightness of the checked pixel the same as the darkest pixel from a starting point to that pixel. 3. The position recognition means extracts data in the vicinity of a predetermined part of the driver in the background-processed image data and data of a standard pattern of the predetermined part stored in advance, and determines the position of the data of the standard pattern in the image data. 3. A vehicle driver position recognition device according to claim 1 or 2, wherein the position of the predetermined part is determined as the position of the predetermined part where the correlation obtained by comparing and comparing the above changes is determined. 4. A vehicle driver position recognition device according to any one of claims 1 to 3, wherein the predetermined part is the driver's nose. 5 Claim 1 in which the light emitting means emits infrared light and the image detection means is configured to receive infrared light.
A device for recognizing the position of a vehicle driver according to any one of items 1 to 4. 6. The position of the vehicle driver according to any one of claims 1 to 5, wherein the image detection means is arranged on the passenger seat side of the vehicle so as to detect the riding state of the vehicle driver as a side image. recognition device. 7. A vehicle driver position recognition device according to any one of claims 1 to 6, wherein the image detection means comprises a two-dimensional solid-state image sensor.