JP3036319B2 - Driver status monitoring device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an on-vehicle driving vehicle condition monitoring apparatus which is mounted on a vehicle and automatically recognizes the state of the driver's eyes, and issues an alarm when the driver falls asleep to prevent an accident. It is about.

[0002]

2. Description of the Related Art FIG. 3 illustrates the configuration of a conventional on-vehicle driving condition monitoring apparatus. In FIG. 3, an imaging unit 1 is an imaging unit, and captures an image of a face portion of a driving vehicle. The recognition device 2 converts the analog imaging data captured by the imaging unit 1 into digital imaging data, and binarizes the digital imaging data with a preset threshold. The digital image data is a set of pixels having a certain luminance value. By performing binarization processing, a white portion where the luminance value of this pixel exceeds the threshold value is "1", and a black portion which does not exceed the threshold value is "1". (Hereinafter, referred to as a black pixel area) is set to “0” and “
The position and state of the eyes in the digital image data are detected from the position and size of the black pixel area.The recognition device 2 uses the detection result to determine whether the driver has fallen asleep or not. When inattentiveness or the like is recognized, a warning is issued from the speaker to the driver.

As described above, in the conventional in-vehicle driver monitoring device, when the driver falls asleep or looks aside, a warning is issued from the speaker.

[0004]

However, in the above-described conventional on-vehicle driver monitoring apparatus, if the digital image data of the driver is binarized to detect the position and state of the eyes, the threshold value setting method Even if the entire area around the eyes becomes a black pixel area or the threshold setting is appropriate, the difference in brightness of the face due to individual differences in the brightness of the vehicle and the shape of the driver causes the eyes to change. However, there is a problem that the position and the state cannot be determined correctly.

SUMMARY OF THE INVENTION The present invention solves the above-mentioned conventional problems, and is an excellent vehicle-mounted driver monitoring system in which the condition of the driver's eyes is correctly determined and a warning is quickly given when the driver falls asleep or looking aside. It is intended to provide a device.

[0006]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides an in-vehicle driver monitoring device which includes an image pickup unit for picking up a driver's face and an image picked up by the image pickup unit. Edge detecting means for detecting a boundary changing from a bright area to a dark area and a boundary changing from a dark area to a bright area, a rectangular area address generating section for generating an address of a rectangular area for limiting an image area, and the edge detecting section Storage means for storing the edge image in the area specified by the rectangular area address generator with respect to the edge image detected by the image processing apparatus, and image data in various states of the driver's eyes are stored. A matching unit that compares the image data sequentially selected from with the edge image stored in the storage unit and measures the degree of coincidence; As well as the cross-sectional, based on the collation position
Finger to narrow the rectangular area with respect to the rectangular address generator
And a controller for sending an instruction to issue a warning to the driver when the degree of coincidence exceeds a predetermined standard.

[0007]

According to the present invention, edge detection is performed on an image of a driver's face portion by the above means to clarify the contour of the driver's face portion, and this edge image is converted into a rectangle including the driver's eye portion. It is extracted to an area, and several kinds of image data of various states of the eye prepared in advance are matched with the extracted edge image to measure the degree of coincidence, and the degree of coincidence does not exceed a preset threshold value In this case, the state of the driver's eyes is accurately grasped by changing to another image data and narrowing the area from which the edge image is extracted.

[0008]

An embodiment of the present invention will be described below with reference to the drawings.

In FIG. 1, an A / D converter 4 is a means for converting input analog data into digital data, and converts an analog image input by the imaging unit 1 into a digital image. The captured image memory 5 is storage means for image data, and stores the digital image. The edge detection circuit 6 is a means for detecting a boundary where a bright region changes to a dark region and a boundary where a dark region changes to a bright region (hereinafter referred to as an edge) on the digital image. This edge is detected from the digital image.

An edge image memory 7 is a storage means for image data, and stores an edge image of the digital image in which the edge detection circuit 6 detects an edge. The rectangular area address generator 8 is a means for generating an address of a rectangular area for limiting the image area, and generates an address of the rectangular area for the edge image stored in the edge image memory 7. As the address of the rectangular area, the address of the area including the position of the eye with respect to the edge image is set in advance.

The template matching section 9 is a comparing means (hereinafter, referred to as template matching) for comparing image data of an image which is a narrow rectangular area and is set in advance (hereinafter, referred to as template data). The template matching is performed on the edge image in the rectangular area address generated in step 8 by using several types of templates stored in advance.

The control / arithmetic unit 10 stores image data for a certain period of time in an internal memory, and is image data analysis means for measuring a temporal change of the image data. On the other hand, the data is stored for a fixed time, the state of the driver is analyzed from the temporal change of the template, and an instruction to narrow the rectangular area is sent to the rectangular area address generator 8.

The blinking motion analysis unit 11 is provided in the control / calculation unit 10 and has a storage unit therein. The blinking motion analysis unit 11 is a means for analyzing a change in the position of the pupil of the template in which a predetermined time has been accumulated. The eye movement analysis unit 12 is a means for analyzing a change in the state of the eyes of the template which has been stored in the control / calculation unit 10 and has a storage means therein for a predetermined time.

The comprehensive judgment section 13 is a means for judging the driver's state from the input data, and judges the driver's state based on the analysis data from the blinking motion analysis section 11 and the eye movement analysis section 12. The signal input unit 14 is an input unit for inputting data relating to the traveling state of the vehicle from the vehicle body, and outputs this data to the comprehensive judgment unit 13.

In FIG. 2, an edge pattern buffer 1
Reference numeral 5 denotes an image storage unit that stores the edge image in the area specified by the rectangular area address generator 8 with respect to the edge image in the edge image memory 7. The template data storage memory 16 previously stores several types of template data having different eyelid opening / closing states or different line-of-sight directions. The template address selector 17 is a means for selecting from several types of templates, and selects one template data from the template data storage memory 16.

The template pattern buffer 18 is a storage unit for storing template data, and stores the template selected by the template address selecting unit 17 from the template data storage memory 16.

The pattern matching section 19 is a means for determining the degree of coincidence of data with one piece of data. The pattern matching section 19 shifts the edge image in the edge pattern buffer 15 by one pixel unit and stores the template data stored in the template pattern buffer 18. Is obtained, and the position of the template and the degree of coincidence with the highest degree of coincidence with the edge image are output as the pattern matching result.

The pattern matching control unit 20 is a means for calculating the eyelid opening / closing state and the pupil position based on the pattern matching result output from the pattern matching unit 19, and the matching degree of the template matching does not reach a preset reference. In this case, an instruction to select another template is sent to the template address selecting unit 17 in order to perform template matching using another template again, and when the matching degree reaches a preset reference, The data of the eyelid opening / closing state and the position of the pupil calculated from the pattern matching result are sent to the control / calculation unit 10.

Next, the operation of the above embodiment will be described. In FIG. 2, an image of the driver's face captured by the imaging unit 1 is shown.
Na log image data is sent to the A / D conversion unit 4. A /
D converter 4 converts the Anal log image data the sent to the digital image data. The digital image data is stored in the captured image memory 5, and the edge is detected by the edge detection circuit 6. By detecting the edge of this digital image data, the driver's face, eyes, nose,
The outline of the mouth and the like becomes clear.

The detected edge image is stored in the edge image memory 7. The edge image stored in the edge image memory 7 and the edge image in the rectangular area designated by the rectangular area address generator 8 are output to the template matching section 9.

The edge image in the rectangular area is stored in the edge pattern buffer 15 in the template matching unit 9, and the pattern matching unit 19
The template data stored in the template pattern buffer 18 is compared with the template data. That is, the template data is superimposed on the edge image while being shifted one pixel at a time, and the degree of coincidence between the edge image and the template data is obtained. When the position of the template data with respect to the edge image and the degree of coincidence are the highest, the pattern data is output to the pattern matching control unit 20 as a pattern matching result.

When the pattern matching result does not reach the preset reference in the pattern matching control section 20, the pattern matching control section 20 uses another template data. At that time, the pattern matching control unit 20 sends an instruction to select another template to the template address selecting unit 17. The template data storage memory 16 stores several types of template data. When the template address selection unit 17 receives the instruction to select the template data, the template data storage memory 16 stores a different template data from the template data used earlier. And outputs it to the template pattern buffer 18. The pattern matching unit 19 obtains the degree of coincidence with the edge image using the new template data, and outputs it to the pattern matching control unit 20.

Next, in the pattern matching control unit 20,
If the pattern matching result has reached a preset reference, the pattern matching result is output to the control / calculation unit 10. In FIG. 1, the pattern matching result is
The analysis is performed separately by a blinking motion analysis unit 11 and an eye movement analysis unit 12 in the control / calculation unit 10.

The blinking motion analysis unit 11 accumulates the pattern matching results for a predetermined time, and analyzes a change in the position of the pupil of the template from the accumulated pattern matching results. The eye movement analysis 12 accumulates the pattern matching results for a certain period of time, and analyzes a change in the eye state of the template from the accumulated pattern matching results. The analysis results of the blinking motion analysis unit 11 and the eye movement analysis 12 are sent to the overall judgment unit 13. When it is determined that a warning should be issued to the driver including the data on the traveling state of the vehicle from the vehicle body by the signal input unit 14 and the analysis result, the general determination unit 13 issues a warning signal from the speaker 3. Is sent.

As described above, according to the above embodiment, the contour of the driver's face can be clarified by outputting edges to the image of the image pickup unit 1, and a rectangular area is set for extraction of eye movement. When performing the processing with the edge image in the rectangular area and performing the processing, there is an advantage that the precision and processing speed can be increased by narrowing the rectangular area.

[0026]

As is apparent from the above embodiment, the present invention provides an in-vehicle driver monitoring device with an image pickup unit for picking up a driver's face and a bright area on the image taken by the image pickup unit. Edge detection means for detecting a boundary that changes from a dark area to a dark area, and a boundary that changes from a dark area to a light area,
A rectangular area address generating section for generating an address of a rectangular area for limiting an image area, and storing the edge image in the area designated by the rectangular area address generating section with respect to the edge image detected by the edge detecting section. The storage unit stores image data in various states of the driver's eyes, and compares image data sequentially selected from the image data with edge images stored in the storage unit to determine a degree of coincidence. A collating unit to be measured, and the degree of coincidence measured by the collating unit is determined, and the rectangular shape is determined based on the collating position.
An instruction to narrow the rectangular area to the shape address generator
A control unit for sending an instruction to issue a warning to the driver when the degree of coincidence exceeds a predetermined criterion, performing edge detection on the image of the driver's face, In addition to clarifying the outline of the portion, the edge image is extracted into a rectangular area including the driver's eye portion, and several types of image data of various states of the eyes created in advance for the extracted edge image are extracted. When the coincidence does not exceed a preset threshold value, the image data is changed to another image data, and the area from which the edge image is extracted is narrowed, so that the driver's eyes can be obtained. is intended to accurately grasp the state, the accuracy and processing speed of the eye condition of the driver is increased, it has the effect of preventing traffic accidents in drowsy and inattentive, etc. for more reliable driver That.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a vehicle-mounted driver state monitoring device according to an embodiment of the present invention.

FIG. 2 is a block diagram of components of a vehicle-mounted driver status monitoring device according to an embodiment of the present invention.

FIG. 3 is a configuration block diagram of a conventional vehicle-mounted driver state monitoring device.

[Explanation of symbols]

 Reference Signs List 1 imaging unit 5 captured image memory 6 edge detection circuit 8 rectangular area address generation unit 10 control / calculation unit 15 edge pattern buffer 16 template data storage memory 17 template address selection unit 19 pattern matching unit 20 pattern matching control unit

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G08G 1/16 G06T 1/00 G08B 21/00 H04N 7/18

Claims (2)

(57) [Claims] 1. An image pickup unit for picking up a driver's face, and an edge for detecting a boundary where a bright region changes to a dark region and a boundary where a dark region changes to a bright region on an image captured by the image pickup unit. Detecting means, a rectangular area address generating section for generating an address of a rectangular area for limiting an image area, and an edge in an area designated by the rectangular area address generating section with respect to the edge image detected by the edge detecting section. A storage unit for storing an image and image data in each state of the driver's eyes are stored, and the image data sequentially selected from the image data is compared with an edge image stored in the storage unit. When the collating unit that measures the degree of coincidence and the degree of coincidence measured by the collating unit are determined ,
In both cases, the rectangular address generation unit
A control unit for transmitting an instruction to narrow the rectangular area, and sending an instruction to issue a warning to the driver when the degree of coincidence exceeds a predetermined reference. Condition monitoring device. 2. A collating unit comprising: an image storage unit for storing an image; a storage memory for storing image data in various states of a driver's eyes; and an address selection unit for sequentially selecting from the storage memory. A pattern matching unit that measures the degree of coincidence between the image data selected by the address selection unit and the image stored in the image storage unit, and determines the degree of coincidence measured by the pattern matching unit.
If this determination result is a value that is less than a preset threshold value, it is output. If this determination result is greater than the threshold value, an instruction to select one of several types of image data in the storage memory is sent to the address selection unit. The in-vehicle driver status monitoring device according to claim 1, further comprising a pattern matching control unit.