JP2004070411A - Driver monitor device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driver monitor device capable of easily detecting the status of a driver by means of a simple arrangement. <P>SOLUTION: The driver monitor device is mounted in a vehicle to monitor the status of a driver based on an image of the driver's face. The device comprises an image pickup means 1 for picking up images of the driver's face; an image storage means 2 for storing in time series the face images picked up by the image pickup means 1 at predetermined sampling intervals; a differential computing means 3 for computing differences in density at each pixel among the face images stored in time series; and a movement detecting means 5 for calculating the number of pixels where density has varied based on the differential computing means 3 and for determining that the driver has moved his face when the number of pixels is equal to or greater than a predetermined reference value. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a driver monitoring device that detects a driving state of a driver of a vehicle.
[0002]
[Prior art]
As a conventional driver state detecting device, for example, a device described in Japanese Patent Application Laid-Open No. H11-161798 (hereinafter, referred to as a conventional example) is known. In the conventional example, a time difference between driver's face images sequentially photographed by a camera is calculated, a portion where the difference value is equal to or more than a predetermined value is cut out as a face position, and the direction of the driver's face is determined based on the cut out face image. To detect. Then, based on the detection result, the driver's inattentiveness or consciousness reduction is determined.
[0003]
[Problems to be solved by the invention]
However, in the above-described conventional example, since the direction of the face is detected by using a neural network with respect to the image of the cut out face portion, complicated and various operations are required, and furthermore, learning is performed. However, there is a problem that a large-capacity memory is required and the device configuration becomes complicated.
[0004]
The present invention has been made to solve such a conventional problem, and an object of the present invention is to provide a driver monitor device having a simple configuration and capable of easily detecting a driver state. It is in.
[0005]
[Means for Solving the Problems]
In order to achieve the above object, the present invention provides a driver monitoring device mounted on a vehicle and monitoring the state of the driver based on an image of the driver's face portion, wherein the image capturing means captures the driver's face, An image storage means for storing the face images captured at predetermined sampling times by the image capturing means in time series, and a density difference for each pixel between the face images stored in time series. Difference calculating means, and motion detecting means for determining the number of pixels having a density difference from the difference calculating means, and determining that the driver has moved the face when the number of pixels is equal to or greater than a predetermined reference value. And characterized in that:
[0006]
【The invention's effect】
According to the present invention, face images taken at predetermined sampling intervals are stored in chronological order, and a difference value between density values of the stored time-series image data is obtained. Is obtained. When the number of pixels exceeds a predetermined value, it is determined that the driver's face is moving. Therefore, it is possible to accurately capture the feature amount of the face movement of the driver while driving, such as the driver's gaze state in front, temporary stoppage, lane change, visual safety confirmation action when turning left, and inattentive action. Driver status can be easily detected. Further, the device configuration can be simplified as compared with the conventional device.
[0007]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the driver monitor device according to the first embodiment of the present invention. As shown in FIG. 1, the driver monitor device 100 includes an image capturing unit 1 that continuously captures a driver's face, and a driver's face image captured at predetermined sampling times by the image capturing unit. Image storage means 2 for storing in time series, difference calculating means 3 for calculating a difference in density which changes in time series for each pixel between face images stored for each predetermined sampling time, Face image recreating means 4 for recreating the driver's face image based on the difference data computed by the computing means 3, and face movement for detecting the driver's face motion based on the recreated face image Detecting means 5.
[0008]
In the present embodiment, a change in the density of each pixel of the image data stored in the image storage unit 2 in time series is detected. If it is larger than the value), it is determined that the driver has moved his / her face.
[0009]
Hereinafter, the operation of the present embodiment will be described with reference to the flowcharts shown in FIGS.
[0010]
First, in step 201 shown in FIG. 2, when the image of the driver's face is captured by the image capturing unit 101, a process of storing image data in the image memory IMAGE_NEW of the image storage unit 3 is performed in step 202. Is
[0011]
Next, the routine proceeds to step 203, where it is checked whether image data is stored in the image memory IMAGE_OLD.
[0012]
Immediately after the start of this processing, since image data is not stored in the image memory IMAGE_OLD, the process proceeds to step 214. In step 214, the image data in the image memory IMAGE_NEW is assigned to the image memory IMAGE_OLD, and the process returns to step 201 to input the next image.
[0013]
In step 202, the newly acquired image data is stored in the image memory IMAGE_NEW. After the above-described processing loop has been performed once, the image data is stored in the image memory IMAGE_OLD. Therefore, the process proceeds to step 204 to calculate the density difference value between the image memories IMAGE_NEW and IMAGE_OLD.
[0014]
The specific processing contents up to this point will be described with reference to FIGS. FIGS. 4 to 8 are explanatory diagrams showing the state of the driver's face and the time-series density change of each pixel included in a partial area “a” of the image captured by the image capturing unit 1. This indicates that the density value of each pixel included in the area “a” shown in (a) to (j) changes in the order of (1) to (10). Also, the difference between the previous density value and the current density value is shown, respectively (for example, the data shown in “(2)-(1)” is the difference between the image data (2) and the image data (1). The difference between the density values is shown).
[0015]
Assuming that the image data to be taken in first is (1) (FIG. 4), the image data of (1) is temporarily stored in the image memory IMAGE_NEW, and is thereafter substituted into the image memory IMAGE_OLD. Then, the image data (2) (FIG. 4) captured secondly is newly stored in the image memory IMAGE_NEW.
[0016]
At this time, the region where the calculation of the density difference value in step 204 is performed will be described as being limited to the region “a” in FIG. The area “a” is an area including 20 pixels and one line in the horizontal direction and including the right end of the face, and is shown under the condition that the absolute position of the image is fixed.
[0017]
The reason why the position is fixed at this position is that the density change is remarkable and is suitable for explanation. In the normal processing, the entire screen is targeted, and when the face position can be specified, the position is determined. An area may be cut out as a target.
[0018]
In addition, the calculation time of the density difference value of the image data of (1) and (2), which is the detection tact of the captured image, that is, the time when the calculation of the density difference of each pixel between the face images is started, and then the face image The time interval between the start of the calculation of the density difference for each pixel is 33 msec / frame when the image capture timing (sampling time) of a general TV camera is set, and the difference calculation is performed during image data capture. Assuming that the process is performed as a back process, the length is 33 msec.
[0019]
If the back processing of the difference calculation is not performed, the calculation time is 33 msec + α. In this case, the calculation time differs depending on the size of the region to be processed. Therefore, the fixed value of the detection tact when the difference calculation is performed in the back processing is 33 msec, and the fixed value of the detection tact when the difference calculation is not performed in the back processing is to add a processing time proportional to the size of the processing area. It will be something.
[0020]
When performing processing for the entire screen with a hardware configuration that cannot perform back processing of the difference operation, as a countermeasure when the fixed detection tact is too long to detect the fast movement of the driver, Alternatively, a method of shortening the detection tact by performing the processing only on the face width or the eye region can be used.
[0021]
Next, the method of calculating the density difference value in step 204 in FIG. 2 will be described using the image data (1) and (2) in FIG. Each square shown in FIG. Each pixel in the area “a” is shown, and the number described in this square indicates the density value of that pixel. At this time, the density value is a gray value of 256 gradations, where 0 is black and 255 is white. The calculation result of the density difference value in step 204 is obtained by subtracting the density value of the image data of (1) from the density value of the image data of (2) in the square indicated by (2)-(1). It is described in.
[0022]
Thus, each time new image data is taken in, the image memories IMAGE_NEW and IMAGE_OLD are updated, and the calculation of these difference values “(3)-(2), (4)-(3),...” Is sequentially performed. Repeated.
[0023]
In step 205, it is confirmed whether or not image data has been substituted into the difference image memories DIFF_IMAGE1 to DIFF_IMAGE5. Then, the difference image memory DIFF_IMAGE included in the image storage means 102 is shown in “(2)-(1), (3)-(2), (4)-(3),...” In FIGS. Data obtained by taking the absolute value of the density difference data is stored. As a result, difference data as shown by I to IX in FIG. 9 can be obtained. Here, the difference data shown in FIG. 9 is obtained by taking the absolute value of each difference data shown in FIGS.
[0024]
In the present embodiment, the number of difference image memories DIFF_IMAGE is set to “5”, and a density change is detected based on the sum of these (see FIG. 10 described later). Note that, here, the description will be made in a state where the accumulation number of density values is fixed at “5”, but this number is changed according to the length of the detection tact, the density state of the image, and the environment in which the vehicle is running. You can also.
[0025]
Until the image data is substituted into the difference image memories DIFF_IMAGE1 to DIFF5 in Step 205, the processing from Step 201 is repeated through Steps 206 and 214, and the image data is substituted into the difference image memories DIFF_IMAGE1 to 5 In step 207, the difference image memory is updated.
[0026]
In this processing, old data is ejected while new data is taken in one by one into five DIFF_IMAGEs for accumulating density differences. In the first processing, DIFF_IMAGE1 includes "[2]-" shown in FIG. The absolute value data I of the density difference of (1) is substituted. In the subsequent processing, the absolute value data II of the density difference of (3)-(2) is written in DIFF_IMAGE2, and the (4)-▲ is written in DIFF_IMAGE3. 3), the absolute value data IV of the density difference of "5-5" in DIFF_IMAGE4, and the absolute value data of the density difference of "6-5" in DIFF_IMAGE5. V are respectively substituted.
[0027]
In step 207, the II data substituted for DIFF_IMAGE2 is sequentially updated to DIFF_IMAGE1 and the III data substituted for DIFF_IMAGE3 is sequentially updated to DIFF_IMAGE2, and the latest density difference absolute value data VI is substituted for DIFF_IMAGE5. .
[0028]
In step 208 shown in FIG. 3, the sum of the absolute values of the density differences for each pixel stored in DIFF_IMAGE1 to DIFF_IMAGE5 is calculated. The content of this calculation will be specifically described using a density value. In the state where the accumulation number of the absolute value of the density difference of each pixel shown by I to IX in FIG. 9 is fixed to “5”, as shown in FIG. 10, in the first processing, II + III + IV + V + VI, and in the second processing, III + IV + V + VI + VII ,..., Are calculated each time a new image is captured.
[0029]
In step 209, a process is performed in which pixels whose total sum calculated in step 208 is equal to or higher than a predetermined gradation are detected, an image is re-created based on the detected pixels, and the image is displayed.
[0030]
When the predetermined gradation is set to 50 gradations for the area “a” shown in FIG. 4, the hatched pixels in FIG. 10 are detected, and an image is re-created based on these pixels. Will be.
[0031]
In step 210, it is determined whether or not the number of pixels detected in step 209 is within a predetermined range, and if the number of pixels having a predetermined gradation (in this case, 50 gradations) is within this range. Moves to step 211 to determine that the vehicle is in the forward gaze state. In other words, when the number of pixels at which the density difference is equal to or more than 50 gradations is large, it is determined that the driver's face has moved significantly. Conversely, when the number of pixels is small, the driver's face has not moved significantly. In this case (NO in step 210), it is determined that the driver is gazing forward.
[0032]
The determination principle in steps 210 and 211 will be described with reference to FIG. When the vehicle is running, the position of the face is constantly moving due to the vertical movement caused by the minute vibration of the vehicle. In order to effectively capture this facial movement, it is preferable to accumulate the number of display pixels over a predetermined number of processed images. In a state where the driver is gazing forward and driving, as shown in FIG. , The contours of the face and the parts corresponding to the eyes, nose, and mouth are displayed in an emphasized manner. Therefore, the driver's forward gaze state can be determined by using the number of display pixels constituting this area as the determination target.
[0033]
FIG. 11 schematically shows a face image obtained by the above-described image re-creating process in a state where the engine of the vehicle is running and the driver of the vehicle faces his / her front. In the face image that was actually recreated, the width of the outline of the face, head, eyes, nose, mouth, and ears was the width corresponding to the amplitude of the micro vibration of the vehicle. Is configured.
[0034]
In FIG. 11, the driver's face is indicated by a black line for convenience of description, but is actually displayed by a white line on a black background.
[0035]
In the determination based on the number of display pixels in the difference processing only between two frames not based on the processing described in this flowchart (that is, in the determination not using the method of calculating the sum of five times shown in FIG. 10), the hatching in FIG. May be detected as a target image having a large density change (for example, “[6]-[5]”), and if no target pixel exists (for example, “ (7)-(6) ”) also occurs, and it is not easy to continue stabilizing the determination. Note that the pixels indicated by diagonal lines in FIG. 9 correspond to the fact that the accumulation number of the density difference is 5 and the judgment value is 50 gradations in FIG. I have.
[0036]
In step 210 shown in FIG. 3, if the number of display pixels (that is, the number of pixels exceeding 50 gradations) is not within the predetermined range, the process proceeds to step 212, and the number of pixels detected in step 209 is reduced to the predetermined value. Is determined. If the value exceeds the predetermined value, the process proceeds to step 213 to determine that the vehicle is in the inattentive state.
[0037]
At this time, the movement of the driver's face, the pixel value of the area “b” obtained for each tact, and the difference data of the pixel value obtained for each tact are shown in FIGS. (A) to (j) shown in FIGS. 12 to 16 correspond to (a) to (j) in FIGS. 4 to 8 described above, and FIGS. 12 to 16 show data when the driver turns his / her face sideways.
[0038]
FIG. 17 corresponds to FIG. 9, and FIG. 18 corresponds to FIG. From FIG. 17, it is understood that when the driver turns his / her face sideways, the difference data of the density value of each pixel is larger than when the driver is gazing forward. From FIG. 18, the sum of the difference data similarly has a larger value than when the driver gazes ahead.
[0039]
The principle of the determination at this time will be described with reference to FIG. When the driver moves his / her face to look at something other than the front, as shown in the figure, the area emphasized with the movement of the face increases. Therefore, the driver's inattentive state can be determined by using the number of display pixels constituting this area as the determination target.
[0040]
This FIG. 19 also schematically shows an actually recreated image, similarly to FIG. 11, and in the actually recreated image, all the trajectories of the contour lines as shown in FIG. 11 are displayed. It becomes an image.
[0041]
The number of display pixels in FIG. 19 is larger than the number of display pixels in FIG. 11, and the difference between the numbers of display pixels is so large that the digits differ by one or two digits. Therefore, the reference value (predetermined value) used in the determination in step 212 of FIG. 3 is set to a value substantially intermediate between the number of display pixels shown in FIG. 11 and the number of display pixels shown in FIG. It can be determined whether or not the driver has moved his / her face by the determination of.
[0042]
Specifically, the market value of the number of display pixels when the driver's face continues to face the front of the vehicle and the market value of the number of display pixels when the driver turns the face from the front of the vehicle to the side of the vehicle are determined in advance. It is sufficient to set a value which is approximately intermediate between these market values as a reference value. Alternatively, when the driver gets on the vehicle, the number of display pixels in a state where the driver's face continues to face the front of the vehicle may be detected, and the reference value may be set based on the detection result.
[0043]
According to such a configuration, an appropriate reference value can be set, and the movement of the driver's face can be detected with higher accuracy.
[0044]
Further, if the detection tact is adjusted when the image captured by the image capturing unit 1 is stored in the image storage unit 2, it is possible to detect the driver state with high accuracy according to the driving situation. .
[0045]
Further, if the reference value to be set is changed according to the brightness of the image picked up by the image pickup means 1, it is possible to detect the state of the driver having high robustness to the change in the light environment in the vehicle compartment. .
[0046]
In addition, if the reference value to be set is changed according to the driving environment of the vehicle, such as during the daytime, at nighttime, when the weather is fine, when it is raining, etc., more accurate driver state detection adapted to the driving environment can be performed. Becomes possible.
[0047]
Also, in the flowcharts shown in FIGS. 2 and 3, as shown in FIGS. 11 and 19, the configuration is such that the face direction of the driver is detected by increasing or decreasing the detection area for all pixels. In the case of performing the face direction detection, as shown in FIG. 20, by detecting pixels targeted for arbitrary lines in the horizontal direction and detecting the continuity of the pixels in the vertical direction, This can also be performed as the number of detection lines in the vertical direction.
[0048]
That is, the number of pixels (the number of lines) in which pixels exceeding a predetermined number of gradations (50 gradations) are continuously generated in the vertical direction is obtained, and it is determined whether or not this number is within a predetermined range. Can be detected whether the user is gazing forward or looking aside.
[0049]
Next, a driver monitor device according to a second embodiment of the present invention will be described. The configuration of the device is the same as that shown in FIG. Hereinafter, the operation of the second embodiment will be described with reference to the flowcharts shown in FIGS.
[0050]
In the present embodiment, when a time difference is calculated for each pixel, the number of display pixels is accumulated over a predetermined processed image for pixels having a predetermined gradation or higher. Then, the driver's forward gaze state and inattentive state are determined as in the first embodiment.
[0051]
The processing contents in each step shown in FIGS. 21 and 22 are substantially the same as those shown in FIGS. 2 and 3 described above, and only different portions will be described below.
[0052]
In step 1205 shown in FIG. 21, it is determined whether or not the density difference value is equal to or greater than a predetermined gradation for each pixel of the image memories IMAGE_NEW and IMAGE_OLD calculated in step 1204, and a pixel having the predetermined gradation or more is specified. Thereafter, in step 1206, it is determined whether or not image data has been assigned to the difference image memories DIFF_IMAGE1 to DIFF_IMAGE5.
[0053]
In the initial state, no image data is assigned to DIFF_IMAGE, and therefore, in step 1207, the pixels specified in step 1205 are sequentially assigned to the difference image memories DIFF_IMAGE1 to DIFF_IMAGE5.
[0054]
The situation at this time will be described with reference to FIG. FIG. 23 shows data of the density difference values (2)-(1), (3)-(2),..., (10)-(9) in the area "a" shown in FIG. ing. In the present embodiment, the determination reference value in step 1205 is set to be greater than or equal to +10 gradations and smaller than or equal to −10 gradations, and the density difference values satisfying the conditions are indicated by hatching.
[0055]
In the first process, the image data I including the pixels specified by the density difference values of (2)-(1) shown in FIG. 23 is substituted into DIFF_IMAGE1. In the subsequent processing, the image data II composed of the pixel specified by the density difference value of (3)-(2) is substituted for DIFF_IMAGE2, and the image data II specified by the density difference value of (4)-(3) is set for DIFF_IMAGE3. Image data III consisting of pixels, image data IV consisting of pixels specified by density difference values of (5)-(4) in DIFF_IMAGE4, and pixel data specified by density difference values of (6)-(5) in DIFF_IMAGE5 Are respectively substituted.
[0056]
In step 1208, the II data substituted for DIFF_IMAGE2 is moved to DIFF_IMAGE1, the III data substituted for DIFF_IMAGE3 is sequentially moved to DIFF_IMAGE2, and is composed of the pixels specified by the latest density difference value. The image data VI is substituted for DIFF_IMAGE5.
[0057]
In step 1209 shown in FIG. 22, the sum of the pixels specified by the density difference values stored in DIFF_IMAGE1 to DIFF_IMAGE5 is detected as the number of display pixels of the re-created processed image. A method of calculating the sum of the pixels specified by the density difference value will be specifically described using density value data shown in FIG.
[0058]
In the first processing, a pixel having a gradation of 10 or more or −10 or less is detected in the pixel data of DIFF_IMAGE1 (II in FIG. 23), and similarly, the gradation is calculated in DIFF_IMAGE2 (III in FIG. 23). Pixels that are greater than or equal to 10 or less than or equal to −10 are detected (in this case, there are no corresponding pixels). Similarly, in DIFF_IMAGE3 (IV), DIFF_IMAGE4 (V), and DIFF_IMAGE5 (VI), pixels having a gray scale of 10 or more or -10 or less are detected, and the number of pixels that are the sum of these is obtained. However, if the same pixel is the target, no duplicate counting is performed.
[0059]
In step 1210 of FIG. 22, it is determined whether or not the number of pixels detected in step 1209 is within a predetermined range. If it is within the predetermined range, the process shifts to step 1211 to determine that the vehicle is in the forward gaze state.
[0060]
If it is determined in step 1210 that the number of display pixels is not within the predetermined range, the process proceeds to step 1212, where it is determined whether the number of pixels detected in step 1209 exceeds a predetermined value. If it has, the process moves to step 1213 to determine that it is in an inattentive state.
[0061]
FIG. 24 shows the density value of each pixel when the driver's face moves as described in FIGS. 12 to 16 of the first embodiment. It can be understood from the figure that when the driver's face moves, the number of pixels having a large density value increases.
[0062]
Thus, even when the method according to the second embodiment is used, a highly accurate driver state detection can be performed with a simple configuration, similarly to the above-described first embodiment.
[0063]
Next, a third embodiment of the present invention will be described. FIG. 25 is a block diagram illustrating a configuration of a driver monitor device according to the third embodiment. As shown in the figure, the driver monitor device 200 is different from that shown in FIG. 1 in that a vehicle running state detection means 6 and a face movement judgment reference setting means 7 are mounted. I have.
[0064]
The vehicle running state detecting means 6 acquires information on the running of the vehicle, such as the running speed of the vehicle.
[0065]
The face movement determination reference setting unit 7 performs a process of changing a reference value for determining whether the driver's face is moving based on the data detected by the vehicle running state detection unit 6.
[0066]
Hereinafter, the operation of the present embodiment will be described with reference to the flowcharts shown in FIGS. In the present embodiment, by using the vehicle traveling state signal, it is possible to further improve the accuracy of determining the driver's forward gaze state and inattentive state.
[0067]
The description of the processing contents in each step shown in FIGS. 26 to 28 will be made only for parts different from the first embodiment shown in FIGS.
[0068]
In step 1610, it is determined whether the reference display pixel number when the driver gazes forward has been learned. As shown in FIG. 11, when the vehicle is in a running state and the driver is gazing at the front of the vehicle as shown in FIG. Indicates an area value that can be detected by being emphasized. The learning of the reference pixel number is performed in each step shown in the flowchart of FIG.
[0069]
That is, it is determined in step 1701 whether the steering wheel operation amount of the vehicle is small, or not, in step 1702, whether the vehicle is running at a predetermined vehicle speed or more. When these conditions are satisfied, the driver looks ahead. It is determined that there is a high possibility of being in the state, and the reference display pixel number when the driver gazes forward is learned in step 1703.
[0070]
In step 1704, it is determined whether or not the learning amount has reached a specified value. If the learning amount has not been reached, the same loop is repeated to continue learning the reference display pixel number when the driver gazes forward.
[0071]
If it is determined in step 1704 that the learning amount has reached the specified amount, the process proceeds to step 1705 to set a learning completion flag for the face movement detection reference value.
[0072]
In the subsequent flow, the processing shifts from step 1601 to step 1611 in FIG. In the present embodiment, the driver detects a state that requires safety confirmation, such as changing lanes, turning left, or temporarily stopping, by confirming the vehicle traveling state signal. In step 1616, the number of display pixels detected in step 1609 is determined by By judging whether or not the value has been exceeded, it is possible to detect whether or not the driver has performed the safety confirmation (Step 1617) or not (Step 1618) when the driver needs to confirm the safety.
[0073]
If it is determined in step 1611 that the vehicle is not in the vehicle traveling state that requires safety confirmation, the process proceeds to step 1612 to determine the driver's forward gaze state and inattentive state, as in the first embodiment.
[0074]
As described in the flowcharts of FIGS. 26 to 28, by learning the reference display pixel number in the forward gaze state, the face movement corresponding to the individual difference including the case of wearing glasses or sunglasses is learned. Since the reference value can be used as the detection reference value, the state detection accuracy of the driver can be further improved. Further, in addition to the detection of the forward gaze state and the inattentive state, the execution of the safety check can also be detected.
[0075]
In addition, at the initial stage of the start of traveling, since the feature amount based on the individual difference of the driver is learned, the state of the driver having high robustness to the individual difference can be detected.
[0076]
In the first to third embodiments, it is determined whether or not the driver has moved his / her face based on whether or not the number of pixels having a density difference has exceeded a reference value. On the other hand, after an image is created as shown in FIGS. 11 and 19, the display area in the image, that is, the area where the density difference has occurred is calculated by image processing, and the calculated area exceeds the area reference value. Whether or not the driver has moved his / her face may be determined based on whether or not the face has been moved.
[0077]
That is, the driver state may be determined not by calculating the number of pixels having a density difference but by calculating the area of the face image where the density difference has occurred.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a driver monitor device according to a first embodiment and a second embodiment of the present invention.
FIG. 2 is a first partial diagram of a flowchart illustrating a processing operation of the driver monitor device according to the first embodiment;
FIG. 3 is a second partial diagram of a flowchart showing a processing operation of the driver monitor device according to the first embodiment.
FIG. 4 is a first explanatory diagram showing a driver's face image at the time of forward gaze, a density value of each pixel in an area “a”, and a density difference.
FIG. 5 is a second explanatory diagram illustrating a driver's face image at the time of forward gaze, a density value of each pixel in an area “a”, and a density difference.
FIG. 6 is a third explanatory diagram illustrating a driver's face image at the time of forward gaze, a density value of each pixel in an area “a”, and a density difference.
FIG. 7 is a fourth explanatory diagram illustrating a driver's face image at the time of forward gaze, a density value of each pixel in an area “a”, and a density difference.
FIG. 8 is a fifth explanatory diagram showing a driver's face image at the time of forward gaze, a density value of each pixel in an area “a”, and a density difference.
FIG. 9 is an explanatory diagram showing a density difference of a density value detected for each tact at the time of forward gaze.
FIG. 10 is an explanatory diagram showing a total sum of density difference data for five times during forward gaze.
FIG. 11 is an explanatory diagram showing a re-created image at the time of forward gaze.
FIG. 12 is a first explanatory diagram illustrating a face image of a driver in an inattentive state, a density value of each pixel in a region “b”, and a density difference.
FIG. 13 is a second explanatory diagram illustrating a face image of a driver in an inattentive state, a density value of each pixel in a region “b”, and a density difference.
FIG. 14 is a third explanatory diagram illustrating a face image of a driver in an inattentive state, a density value of each pixel in a region “b”, and a density difference.
FIG. 15 is a fourth explanatory diagram illustrating a face image of a driver in an inattentive state, a density value of each pixel in a region “b”, and a density difference.
FIG. 16 is a fifth explanatory diagram illustrating a face image of a driver in an inattentive state, a density value of each pixel in a region “b”, and a density difference.
FIG. 17 is an explanatory diagram showing a density difference between density values detected for each tact in an inattentive state;
FIG. 18 is an explanatory diagram showing a total sum of density difference data for five times in an inattentive state;
FIG. 19 is an explanatory diagram showing a re-created image in an inattentive state;
FIG. 20 is an explanatory diagram showing how a detection line is set on a plurality of lines.
FIG. 21 is a first partial diagram of a flowchart showing a processing operation of the driver monitor device according to the second embodiment.
FIG. 22 is a second partial diagram of the flowchart showing the processing operation of the driver monitor device according to the second embodiment.
FIG. 23 is an explanatory diagram showing density difference data of each pixel when gazing forward, according to the second embodiment.
FIG. 24 is an explanatory diagram showing density difference data of each pixel in an inattentive state according to the second embodiment.
FIG. 25 is a block diagram illustrating a configuration of a driver monitor device according to a third embodiment of the present invention.
FIG. 26 is a first partial diagram of a flowchart showing the processing operation of the driver monitor device according to the third embodiment.
FIG. 27 is a second partial diagram of the flowchart showing the processing operation of the driver monitor device according to the third embodiment.
FIG. 28 is a third partial diagram of the flowchart showing the processing operation of the driver monitor device according to the third embodiment.
[Explanation of symbols]
1 Image capturing means
2 Image storage means
3 Difference calculation means
4 Face image re-creating means
5 Face movement detection means
6 Vehicle running state detecting means
7 Face movement judgment standard setting means
100, 200 driver monitor device

Claims (9)

A driver monitoring device mounted on the vehicle and monitoring the state of the driver based on an image of the driver's face,
Image capturing means for capturing the driver's face;
Image storage means for storing the face images imaged at predetermined sampling times by the image imaging means in time series;
Difference calculation means for calculating the density difference of each pixel between the face images stored in time series,
From the difference calculation means, determine the number of pixels in which a density difference has occurred, and when the number of pixels is equal to or greater than a predetermined reference value, motion detection means for determining that the driver has moved the face,
A driver monitor device comprising: 2. The motion detection unit according to claim 1, wherein the difference calculation unit determines whether the driver has moved a face based on a plurality of pieces of density difference data of the same pixel obtained in time series. 3. Driver monitor device. 3. The reference value according to claim 1, wherein the reference value is set based on a calculation result of a density difference obtained by the difference calculation unit when the driver's face faces forward of the vehicle. A driver monitor device according to any one of the above. 3. The driver monitor device according to claim 1, wherein a reference value for determining whether the driver's face has moved is a variable value. 5. The driver monitor device according to claim 4, wherein the reference value is changed based on brightness of a face image captured by the imaging unit. 6. The driver monitor device according to claim 4, wherein the reference value is changed according to a traveling environment of the vehicle. The difference calculation means is a detection tact that is a time interval from a time when the calculation of the density difference for each pixel between the face images is started to a time when the calculation of the density difference for each pixel is next started between the face images. The driver monitor device according to any one of claims 1 to 6, wherein is appropriately changed. The means for detecting the number of display pixels of the re-created processed image includes a line when a density value exceeds a predetermined value, on a plurality of vertical lines, for a plurality of lines set in the horizontal direction of the image. The driver monitor device according to claim 1, wherein the detection is performed based on a number. 9. The vehicle according to claim 1, further comprising: a traveling state detection unit configured to detect a traveling state of the vehicle, wherein the reference value is set based on a detection result of the vehicle state detection unit. A driver monitor device according to claim 1.

Images