JP2004347464A - Looking-aside detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a looking-aside detection device capable of improving accuracy on detection of a looking-aside action of a driver. <P>SOLUTION: A tracking part 22 tracks the eye position of the driver based on an imaged image including the driver's face imaged by an imaging part 10. A learning part 23 determines the moving quantity and a residence time of the eye position outputted in time series from the tracking part 22, and learns the eye position on the imaged image when the driver looks at a room mirror or the like. An image region setting part 24 sets a prescribed number of image regions having different looking-aside allowable times based on the learned eye position on the imaged image. A looking-aside detection part 25 performs looking-aside detection based on the looking-aside allowable times allocated on each image region. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an inattentiveness detection device.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, an inattentiveness detection device that detects an inattentive behavior of a driver based on an allowable inattentive time according to a driver's face angle and a time during which the driver's face angle is maintained is known (for example, Patent Document 1). , 2).
[0003]
In this device, the act of looking aside is changed according to the direction of the driver's face to determine the act of looking aside. For example, when the driver's face angle is large, it is difficult for the driver to see the scenery in the forward direction, etc., so the inattentive act is determined by setting the inattentive time to be short. In addition, when the driver's face angle is small, the scenery in the forward direction is relatively easy to enter the driver's field of view.
[0004]
As described above, the driver's inattentive behavior is suitably detected by changing the inattentive time.
[0005]
[Patent Document 1]
JP-A-3-16798
[0006]
[Patent Document 2]
JP-A-7-57172
[0007]
[Problems to be solved by the invention]
However, the inattentiveness detection device does not take into account left and right pillars that obstruct the driver's field of view, and is therefore insufficient to appropriately determine inattentive behavior. For example, even when the face direction angle is small, if the driver's visual recognition position is near the pillar portion or the like, the driver's view is blocked, so the allowable inattentive time is relatively short. And so on.
[0008]
As described above, the conventional inattentiveness detection device has not yet been able to properly detect the inattentive behavior of the driver.
[0009]
[Means for Solving the Problems]
According to the present invention, the imaging unit captures the driver's face of the vehicle, and the tracking unit tracks the position of the driver's eyes based on the captured image including the driver's face obtained by the imaging unit, The learning unit determines the movement amount and the dwell time of the eye position output in time series from the tracking unit, and learns the position of the eye on the captured image when the driver looks at the predetermined position. The area setting means sets a predetermined number of image areas having different allowable inattentive times based on the positions of the eyes on the captured image learned by the learning means, and the inattentive detection means allocates the image areas to the image areas set by the image area setting means. An inattentive detection is performed based on the determined inattentive time.
[0010]
【The invention's effect】
According to the present invention, a predetermined number of image areas having different allowable inattentive times are set based on the position of the eye on the captured image. Usually, a driver visually recognizes a relatively large number of structures and the like inside the vehicle. Therefore, by using the position of the eye on the captured image as a reference, an image area can be set based on the position of a structure or the like inside the vehicle. In addition, since it is based on the position of the structure inside the vehicle, a permissible inattentive time taking into account a portion that blocks the driver's view can be allocated to each of the predetermined number of image areas.
[0011]
Then, inattention is detected based on the allowable inattentive time allocated to each image region and the position of the eye to be tracked. For this reason, inattentiveness detection suitable for the in-vehicle environment can be performed on the basis of the inattentive allowance time allocated to each image region.
[0012]
Therefore, it is possible to improve the accuracy of detecting the driver's inattentive behavior.
[0013]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings.
[0014]
FIG. 1 is a configuration diagram of an inattentiveness detection device according to the first embodiment of the present invention. As shown in FIG. 1, the inattentiveness detection device 1 of the present embodiment includes an imaging unit (imaging unit) 10 for imaging the face of the driver of the vehicle. The imaging unit 10 is, for example, a CCD camera for imaging visible light, and is installed below the front of the driver.
[0015]
In addition, the inattentiveness detection device 1 is configured to detect a driver's inattentive act based on a captured image captured by the image capturing unit 10 and, when the processor 20 detects the driver's inattentive act, And an alarm 30 for notifying the driver of the fact.
[0016]
Next, a detailed configuration of the processing device 20 will be described with reference to FIG. FIG. 2 is a functional block diagram of the inattentive detection device 1 according to the first embodiment of the present invention.
[0017]
The processing device 20 includes a position detection unit 21 that detects the position of the driver's eye based on data of the captured image from the imaging unit 10 and a tracking unit (tracking unit) 22 that tracks the position of the driver's eye. And
[0018]
The position detection unit 21 detects the position of the eye from the entire captured image input from the imaging unit 10. In addition, the tracking unit 22 determines a tracking area including the position of the eye on the image with reference to the position of the driver's eye detected by the position detection unit 21, and determines the position of the eye from a later captured image. When detecting, the driver's eyes are detected from within the tracking area.
[0019]
Further, the processing device 20 includes a learning unit (learning unit) 23 that determines the movement amount of the eye position and the staying time that are output in time series from the tracking unit 22. The learning unit 23 learns the position of the eye on the captured image when the driver looks at the left and right door mirrors, the room mirror, and the like, based on the determined moving amount and the residence time.
[0020]
Further, the processing device 20 includes an image region setting unit (image region setting unit) 24 that sets an image region based on the position of the eye on the captured image learned by the learning unit 23. The image area setting unit 24 sets a predetermined number of image areas. Here, the image area is an area to which an inattentive allowable time for detecting inattentiveness is allocated. The image area setting unit 24 allocates different allowable inattentive times to each of the predetermined number of image areas.
[0021]
The processing device 20 includes an inattentiveness detection unit (inattentiveness detection means) 25 that performs inattentiveness detection. The inattentiveness detection section 25 performs inattentiveness detection based on the allowable inattentive time. Specifically, the inattentiveness detection unit 25 determines which image area the position of the eye tracked by the tracking unit 22 belongs to. Then, inattentiveness detection is performed based on the dwell time of the eye position tracked by the tracking unit 22 and the inattentive time of the image area to which the eye position belongs.
[0022]
Next, the outline of the operation of the inattentiveness detection device 1 will be described. First, the imaging unit 10 captures an image of a region including the driver's face, and sends data of the obtained captured image to the processing device 20.
[0023]
The position detection unit 21 of the processing device 20 receives the captured image data and detects the position of the driver's eye. The detection of the position of the eye is performed, for example, as follows. FIG. 3 is an explanatory diagram of the initial processing performed by the position detection unit 21 when detecting the position of the eye. In FIG. 3, a captured image of 480 pixels vertically and 512 pixels horizontally is described as an example.
[0024]
First, the position detection unit 21 acquires density value data for all pixels in the vertical direction of the image. That is, density value data is obtained from the coordinates (0, 0) to (0, 479) shown in FIG. 3, and then density value data is obtained from the coordinates (1, 0) to (1, 479). I do. Then, through Xa and Xb shown in FIG. 3, density value data is finally acquired for the line from coordinates (511, 0) to (511, 479). After that, the position detection unit 21 extracts pixels whose change in density value satisfies a predetermined condition, and obtains a pixel group as shown in FIG. FIG. 4 is an explanatory diagram illustrating a state when the position detection unit 21 extracts a predetermined pixel. As shown in the figure, the extracted pixels correspond to the positions of the driver's eyebrows, eyes, nose, and mouth. More specifically, two pixels A1 and A2 are extracted from the Xc line. As for the xd line, four pixels A1 to A4 are extracted. These pixels A1 to A4 are distinguished by, for example, the amount of change in density value. Then, after the extraction of the pixels, the position detection unit 21 groups the pixels adjacent in the horizontal direction of the image.
[0025]
FIG. 5 is an explanatory diagram illustrating a state in which pixels adjacent in the horizontal direction of the image are grouped. As shown in the drawing, by grouping, the position detection unit 21 forms continuous data G1 to G6 corresponding to the driver's right eyebrow, left eyebrow, right eye, left eye, nose, and mouth, respectively.
[0026]
After that, the position detection unit 21 performs a zoning process. FIG. 6 is an explanatory diagram illustrating a state after zoning by the position detection unit 21. The position detection unit 21 zones the existing positions of the continuous data G1 to G6 in the image vertical direction. At this time, the position detection unit 21 forms three zones (ZONE: L, ZONE: C, ZONE: R). Then, the position detection unit 21 detects the position of the eye by determining the relative positional relationship.
[0027]
After detecting the position of the eye, the tracking unit 22 stores the coordinate value of the detected position of the eye, and sets a tracking area smaller than the entire image based on the stored position. Thereafter, the tracking unit 22 detects the position of the eye in the tracking area every time a captured image is input.
[0028]
FIGS. 7A and 7B are explanatory diagrams illustrating how the tracking unit 22 tracks the position of the eye. FIG. 7A illustrates an initial tracking area, and FIG. 7B illustrates an eye detected after setting the initial tracking area. (C) shows the tracking area newly set based on the detected eye position, and (d) shows the eye position detected from within the newly set tracking area. I have. In FIG. 7, it is assumed that the eye indicated by the broken line has been detected last time, and the eye indicated by the solid line has been detected this time.
[0029]
When the position of the eye is detected by the position detection unit 21, the tracking unit 22 sets a tracking area around the detected position of the eye (FIG. 7A). The coordinate position of the eye at this time is (xk1, yk1). Thereafter, when a captured image is input, the tracking unit 22 detects the position of the eye from within the tracking area centered on (xk1, yk1) (FIG. 7B). The coordinate position of the eye detected at this time is defined as (xk2, yk2).
[0030]
Then, the tracking unit 22 sets a tracking area centered on the coordinate position (xk2, yk2) of the newly detected eye (FIG. 7C). Thereafter, when the captured image is input again, the tracking unit 22 detects the position of the eye from the tracking area centered on (xk2, yk2). Let the coordinate position of the eye detected at this time be (xk3, yk3).
[0031]
Then, the tracking unit 22 sets a tracking area centered on the new eye coordinate position (xk3, yk3) again (FIG. 7D). Thereafter, similarly, the tracking unit 22 detects the position of the eye from within the tracking area.
[0032]
The tracking unit 22 sends data of the detected eye position to the learning unit 23 and the inattentive detection unit 25 as shown in FIG. The learning unit 23 learns eye position data input in time series. Usually, a driver who is driving a vehicle visually checks a room mirror, left and right door mirrors, and the like in order to perform a confirmation operation. For this reason, the learning unit 23 can know the position of the eyes when looking at the positions of the room mirror and the left and right door mirrors by determining the amount of movement of the eyes and the like on the captured image.
[0033]
FIG. 8 is an explanatory diagram showing a state when the driver visually recognizes the front, the rear-view mirror, and the left and right door mirrors. FIG. 8A illustrates the position of the driver's eyes when the driver visually recognizes the front. Is shown. Also, (b) shows the position of the driver's eye when looking at the room mirror, (c) shows the right door mirror, and (d) shows the driver's eye while looking at the left door mirror. The eye position is shown. Further, (e) shows the coordinate position of the eye according to the visually recognized location of the driver. In FIG. 8, an image of 640 pixels in the X direction and 480 pixels in the Y direction will be described as an example.
[0034]
As shown in FIGS. 8A to 8D, when the driver visually recognizes the front, the rear-view mirror, and the left and right door mirrors, as shown in FIG. It is almost constant, except when the user is gazing forward. Specifically, when the driver is looking ahead, the coordinate positions (x, y) of the eyes are dispersed at (260, 220) to (285, 240). Here, the center position of both eyes is set as the coordinate position of the eye.
[0035]
On the other hand, when the driver is looking at the right door mirror, the driver concentrates on (175, 245). When the driver looks on the left door mirror, the driver concentrates on (395, 245). When the driver is looking at the rearview mirror, the driver concentrates on (380, 220).
[0036]
As described above, since the driver's eye position when viewing the mirror is concentrated to a certain extent in the same place, the amount of movement of the eye between the front view and the mirror view can be obtained by collecting data in advance. It can be acquired in a state that includes individual differences. That is, when the eye moves by a predetermined movement amount in a predetermined direction, it can be useful to estimate that one of the three mirrors is about to be visually recognized.
[0037]
Here, the residence time of the driver's eyes when viewing the mirror is known to be 0.7 seconds on average. FIG. 9 is an explanatory diagram showing the relationship between the dwell time of the driver's eyes and the vehicle speed when viewing the mirror.
[0038]
As shown in the figure, it can be seen that the residence time of the driver's eyes is concentrated at a location of about 0.7 seconds. It can also be seen that this time is not affected by vehicle speed. Therefore, it is possible to identify which mirror has been visually recognized based on the moving direction and the moving amount and the dwell time of the eye.
[0039]
As described above, the learning unit 23 learns the position of the eye on the captured image when the driver looks at the mirror based on the direction and amount of movement of the eye and the dwell time of the eye. Then, the learning unit 23 sends data of the learned eye position to the image area setting unit 24.
[0040]
The image area setting unit 24 sets a predetermined number of image areas having different allowable inattentive times based on the eye position data learned by the learning unit 23. That is, an image area is set on the captured image based on the positions of the eyes when viewing the left and right door mirrors and the room mirror.
[0041]
FIGS. 10A and 10B are explanatory diagrams illustrating examples of a predetermined number of image areas having different allowable inattentive times. FIG. 10A illustrates an example of a predetermined number of image areas, and FIG. 10B illustrates an allowable inattentive time of each image area. Is shown. The numerical values in the vertical and horizontal directions shown in FIG. 10A indicate the driver's face angle. The image area is originally set on the captured image, but in FIG. 10, an image area based on the driver's field of view will be described for convenience of description. Further, in the following description, a forward area that the driver most visually recognizes during driving is referred to as a forward gaze area.
[0042]
The image area setting unit 24 sets, for example, five image areas A to E, as shown in FIG. First, the image area setting unit 24 determines the right end of the image area A based on the position of the eye when viewing the room mirror. Further, the image area setting unit 24 determines the left end and the lower end of the image area B based on the position of the eye when viewing the right door mirror, and determines the right end and the lower end of the image area C based on the position of the eye when viewing the left door mirror. Determine the bottom edge. Further, the upper end of the image area E is determined based on the positions of the eyes when the left and right door mirrors are viewed.
[0043]
Further, the image area setting unit 24 determines the left end of the image area D based on the tracking limit position in the right direction. FIGS. 11A and 11B are explanatory diagrams of a setting method of the image area D by the image area setting unit 24. FIG. 11A illustrates a face orientation when the driver looks at the left door mirror, and FIG. The figure shows the face orientation when the driver looks at the right door mirror.
[0044]
As shown in FIG. 11A, the position of the driver's left eye when the driver looks at the left door mirror has almost reached the tracking limit. That is, if the driver turns his / her face to the left any more from the state where he / she is looking at the left door mirror, the left eye will not be imaged. As described above, when the left door mirror is viewed, the limit of capturing the left eye of the driver has been reached.
[0045]
In addition, as shown in FIG. 11B, when the driver is looking rightward, the right eye may reach the tracking limit. For example, there is a case where the driver is looking further to the right than the right door mirror for checking an intersection or the like. In this case, if the driver turns his face to the right any longer, the right eye will not be imaged. This position is the rightward tracking limit position. The image area setting unit 24 specifies a rightward limit position in the captured image, determines the left end of the image area D based on the tracking limit position, and sets the image area D.
[0046]
Note that the tracking limit position in the right direction may be obtained from the captured image as described above, but may be obtained by another method. That is, the tracking limit position in the right direction may be obtained by calculation based on the tracking limit position when the left door mirror is viewed. In this case, if the leftward tracking limit position and the target position are obtained, the rightward tracking limit position is obtained.
[0047]
FIG. 12 is an explanatory diagram of a specific example of the image areas A to E set on the captured image. When the image area shown in FIG. 10A is replaced on the captured image, the result is as shown in FIG. As described above, these areas A to E are set with reference to the coordinate positions of the eyes when viewing the room mirror and the coordinate positions of the eyes when viewing the left and right door mirrors. The left and right tracking limit positions are also used as references.
[0048]
FIG. 10 is referred to again. The image area setting unit 24 sets different allowable inattentive times for each of the image areas A to E allocated as described above. For example, as shown in FIG. 10B, the allowable inattentive time of the image area D is shorter than the allowable inattentive time of the image area B. This is because the face angle of the image area D is larger than that of the image area B. Similarly, the allowable inattentive time of the image area C is shorter than the allowable inattentive time of the image area A.
[0049]
Further, the allowable inattentive time of the image area B is shorter than the allowable inattentive time of the image area A. When both areas A and B are visually recognized, the driver's face angle is almost the same. However, a pillar is provided between the image area B and the front gaze area. When the driver visually recognizes the image area B with the pillar, the front gaze area becomes difficult to see. For this reason, even if the face orientation angles of the two regions A and B are almost the same, the allowable inattentive time of the image region B is shorter than the allowable inattentive time of the image region A.
[0050]
Further, the allowable inattentive time of the image area D is shorter than the allowable inattentive time of the image area C. The face orientation angles of these two regions C and D are almost the same. Further, a pillar is provided between the image area B and the front gaze area via the image area B. Similarly, a pillar is provided between the image area C and the front gaze area. However, the left pillar is located farther from the driver than the right pillar. For this reason, the apparent thickness of the pillar from the driver is different. Then, for the driver, the thicker the apparent pillar thickness becomes, the more obstructive the visibility of the forward gaze area is. Therefore, the allowable inattentive time of the image area D is shorter than the allowable inattentive time of the image area C.
[0051]
Further, the allowable inattentive time of the image area E is substantially the same as the allowable inattentive time of the image area D. For example, when the driver visually recognizes the installation position of the audio device or the like, the front gaze area becomes almost invisible because the eyelids are lowered. For this reason, the image area E has a shorter allowable inattentive time, which is almost equal to the image area D having the shortest allowable inattentive time.
[0052]
The inattentiveness detection unit 25 detects the driver's inattentive behavior based on the image areas A to E and the allowable inattentive time set as described above. In other words, if the inattentive viewing time has elapsed while the position of the eye is within any of the image areas A to E, the inattentiveness detection unit 25 detects the inattentive act of the driver.
[0053]
Next, the operation of the inattentive detection device 1 according to the present embodiment will be described in detail. First, the imaging unit 10 of the inattentiveness detection device 1 captures an image of a region including the driver's face, and sends data of the obtained captured image to the processing device 20.
[0054]
Then, the processing device 20 executes the processing according to FIG. FIG. 13 is a flowchart showing a detailed operation of the processing device 20 shown in FIG.
[0055]
As shown in the figure, first, the position detection unit 21 inputs data of a captured image (ST10). Thereafter, the position detection unit 21 determines whether or not an eye tracking area has been set (ST11). If it is determined that the eye tracking area has been set (ST11: YES), the process proceeds to step ST14.
[0056]
On the other hand, when it is determined that the eye tracking area has not been set (ST11: NO), the position detection unit 21 detects the position of the eye from the entire image (ST12). Here, the position of the eye is detected as described with reference to FIGS. Then, the position detection unit 21 sends data of the position of the eye to the tracking unit 22.
[0057]
After that, the tracking unit 22 sets a tracking area based on the eye position data from the position detection unit 21 (ST13). Then, the tracking unit 22 detects an eye from within the tracking area based on the newly input captured image data (ST14). Thereafter, the tracking unit 22 determines whether or not the eye has been correctly tracked (ST15).
[0058]
When it is determined that the eye tracking is not performed correctly (ST15: NO), the tracking unit 22 clears the eye tracking area (ST16), and the process returns to step ST10. In this case, the tracking area is set again.
[0059]
On the other hand, when it is determined that the eye tracking is correctly performed (ST15: YES), the tracking unit 22 updates the eye tracking area (ST17). That is, as described with reference to FIG. 7, the tracking area is updated based on the position of the eye.
[0060]
Thereafter, the learning unit 23 determines whether the learning of the eye position at the time of visually recognizing the predetermined position is completed (ST18). That is, it is determined whether or not learning of all of the room mirror, the left and right door mirrors, and the tracking limit position in the right direction has been completed. When it is determined that the learning of the eye position at the time of viewing the predetermined position has not been completed (ST18: NO), the learning unit 23 performs a learning process of the eye position (ST19). Then, the process returns to step ST10.
[0061]
Then, when the above process is repeated and the learning of the eye position is completed, “YES” is determined in step ST18. Thereafter, the image area setting unit 24 determines whether an image area has been set (ST20). If it is determined that an image area has been set (ST20: YES), the process proceeds to step ST22.
[0062]
On the other hand, when it is determined that no image area is set (ST20: NO), the image area setting unit 24 sets a predetermined number of image areas and sets different inattentive permissible times for those image areas (ST21). .
[0063]
Thereafter, the inattentiveness detection unit 25 performs an inattentiveness detection process based on the data of the position of the eye from the tracking unit 22 and the data on the image area from the image area setting unit 24 (ST22).
[0064]
If no inattentiveness is detected by this process, the process returns to step ST10. On the other hand, when an inattentiveness is detected, the inattentiveness detection unit 25 sends data indicating that the inattentiveness is present to the alarm 30. Then, the notifier 30 performs a notifying operation (ST23). Thereafter, the process ends.
[0065]
In this way, the driver's inattentive behavior is detected. As described above, in step ST21, a predetermined number of image areas having different allowable inattentive times are set, and in step ST22, inattentive detection is performed. As described with reference to FIG. 10, the inattentive-view allowable time is determined in consideration of the installation position of a pillar or the like, such as a structure in a vehicle, and is more appropriate than a conventional one. Has become. Since the inattentive-point detection is performed based on the appropriate inattentive-point allowable time, a device with high detection accuracy can be obtained.
[0066]
Next, the details of the learning process (ST19) shown in FIG. 13 will be described with reference to FIGS. FIG. 14 is a flowchart showing the details of the eye position learning process (ST19) shown in FIG. 13, and shows the learning process when the room mirror is viewed.
[0067]
As shown in the figure, first, the learning unit 23 determines whether or not the eye position has moved rightward by a predetermined amount M1 (ST30). If it is determined that the position of the eye has not moved rightward by the predetermined amount M1 (ST30: NO), the process proceeds to step ST32.
[0068]
On the other hand, when it is determined that the position of the eye has moved rightward by the predetermined amount M1 (ST30: YES), the learning unit 23 turns on the eye movement flag FL1 and performs another processing described with reference to FIGS. The movement flags FL2 to FL4 are turned off (ST31).
[0069]
Thereafter, the learning unit 23 determines whether or not the eye movement flag FL1 is on and the eye position remains at the position moved by the predetermined amount M1 for a predetermined time (ST32). When it is determined that the eye movement flag FL1 is not on or the eye position has not stayed at the position moved by the predetermined amount M1 for a predetermined time (ST32: NO), the processing illustrated in FIG. 14 ends.
[0070]
Here, if the position of the eye has not moved rightward by the predetermined amount M1 (“NO” in step ST30), the movement flag FL1 is not turned on, so the processing shown in FIG. Will end. Further, even if the vehicle has moved rightward by the predetermined amount M1, if the residence time is short, the processing shown in FIG. 14 ends. The predetermined time in step ST32 is set to, for example, "0.7 seconds".
[0071]
As described above, in the learning process, a process of excluding data for which the eye position data should not be learned is performed, and an image area is not set improperly in a subsequent process.
[0072]
By the way, when it is determined that the eye movement flag FL1 is on and the eye position has stayed at the position moved by the predetermined amount M1 for a predetermined time (ST32: YES), the learning unit 23 sets the eye movement flag FL1 to Turn off (ST33). Then, the learning unit 23 determines whether or not the rectangular small area a is set at the position where the eye is staying.
[0073]
When it is determined that the rectangular small area a is not set at the position where the eye is staying (ST34: NO), the rectangular small area a is set at the staying position (ST35), and the processing shown in FIG. finish.
[0074]
On the other hand, when it is determined that the rectangular small area a is set (ST34: YES), the numerical value of the counter α is counted up (ST36). Here, when the rectangular small area a is set but the eye retention position is not within the rectangular small area a, the small area a is set again in step ST35 or the processing shown in FIG. 14 is ended. (Not shown).
[0075]
Thereafter, the learning unit 23 determines whether or not the numerical value of the counter α has reached a predetermined value (ST37). If it is determined that the value of the counter α has not reached the predetermined value (ST37: NO), the processing shown in FIG. 14 ends.
[0076]
On the other hand, when determining that the numerical value of the counter α has reached the predetermined value (ST37: YES), the learning unit 23 sets, for example, the position of the small area a as the position of the eye when the room mirror is viewed (ST38). Thereby, the learning unit 23 completes the learning of the eye position. Thereafter, the process illustrated in FIG. 14 ends.
[0077]
By the way, in parallel with the execution of the processing shown in FIG. 14, the processing shown in FIGS. 15 to 17 is also executed. FIGS. 15 to 17 are flowcharts showing the details of the eye position learning process (ST19) shown in FIG. 15 shows a learning process when the left door mirror is viewed, FIG. 16 shows a learning process when the right door mirror is viewed, and FIG. 17 shows a learning process when the right rear mirror is viewed.
[0078]
Steps ST40 to ST48 shown in FIG. 15 are the same as steps ST30 to ST38 shown in FIG. Steps ST50 to ST58 shown in FIG. 16 and steps ST60 to ST68 shown in FIG. 17 are also omitted. In the processing of FIGS. 15 to 17, the moving directions and the moving amounts M2 to M4 are different in steps ST40, ST50, and ST60, respectively, and in FIG.
[0079]
Naturally, the small areas b to d and the counters β to θ for counting up are also different.
[0080]
Next, the inattentive judgment (ST22) shown in FIG. 13 will be described in detail. FIG. 18 is a detailed flowchart of the inattentive judgment (ST22) shown in FIG. Note that FIG. 18 illustrates a process of determining whether an eye exists in the image area A. Also, FIG. 18 shows a case where the position of the eye is within the image area A, but the same applies to the other image areas B to E.
[0081]
First, the inattentiveness detection unit 25 determines whether an eye is present in the image area A (ST70). When it is determined that the eyes are present in the image area A (ST70: YES), the inattentiveness detection unit 25 determines whether the timer is operating (ST71).
[0082]
If it is determined that the timer is operating (ST71: YES), the process proceeds to step ST73. On the other hand, if it is determined that the timer is not operating (ST71: NO), the inattentiveness detection unit 25 starts the timer (ST72).
[0083]
Thereafter, the inattentiveness detection section 25 determines whether or not the time measured by the timer (that is, the staying time) exceeds the allowable inattentive time set for the image area A (ST73). When it is determined that the inattentive time has not been exceeded (ST73: NO), the processing illustrated in FIG. 18 ends.
[0084]
On the other hand, when it is determined that the inattentive time has been exceeded (ST73: YES), the inattentiveness detection unit 25 outputs an alarm and resets the timer (ST74). Then, the process illustrated in FIG. 18 ends.
[0085]
If it is determined in step ST70 that the eye is not present in the image area A (ST70: NO), the inattentiveness detection unit 25 resets the timer without outputting an alarm (ST75), and the process ends thereafter. I do.
[0086]
By the way, in the present embodiment, the setting process (ST21) of the image areas A to E is performed by learning about all of the room mirror, the left and right door mirrors, and the rightward tracking limit position (including calculation by calculation). , Is to be executed. However, if at least one learning of the visual recognition position is completed, one of the image areas may be set based on the position and the inattentive judgment may be started. In this case, the image area setting unit 24 sequentially sets the image area each time learning of the position of the eye on the captured image is completed.
[0087]
FIG. 19 is an explanatory diagram showing a process when the image area is set stepwise. As shown in the figure, first, the learning unit 23 determines whether or not the image area A has not been set and the learning of the eye position when the room mirror is viewed is completed (ST80). When it is determined that the image area A has not been set and the learning of the eye position at the time of viewing the room mirror has been completed (ST80: YES), the image area setting unit 24 sets the image area F, and further sets the image area F An inattentive time is set (ST81). Then, the process illustrated in FIG. 19 ends.
[0088]
This image area F is, for example, as shown in FIG. FIG. 20 is an explanatory diagram illustrating an example of an image area set in a stepwise manner. The image area F is an area including the image areas A and C and a part of the image area E. The reason why the image area F is set in step ST81 is that the boundary between the front gaze area and the image area A can be set based on the position of the eye when the room mirror is viewed. Further, the boundary between the image area A and the image area C cannot be set only by the position of the eyes when the room mirror is viewed, so that the image area A includes an area on the side with a larger face direction angle than the image area A. Furthermore, since the lower end of the area cannot be set only by the position of the eyes when the room mirror is viewed, a part of the image area E is included.
[0089]
Description will be made again with reference to FIG. If it is determined that either the image area A has not been set or the learning of the eye position at the time of viewing the room mirror has been completed (ST80: NO), the process proceeds to step ST82. I do. Then, the learning unit 23 determines whether or not the image area C has not been set and the learning of the eye position when the left door mirror is viewed is completed (ST82).
[0090]
When it is determined that the image area C has not been set and the learning of the eye position when the left door mirror is viewed has been completed (ST82: YES), the image area setting unit 24 sets the image area C, and further sets the image area C. The inattentive time of C is set (ST83).
[0091]
The image area C here is as shown in FIG. 20, and is the same as that described with reference to FIG. When the position of the eye at the time of viewing the left door mirror is determined, the right end and the lower end of the image area C are determined as described above. Therefore, the image area setting unit 24 determines the image area C, and further sets the allowable inattentive time.
[0092]
Thereafter, the image area setting unit 24 determines whether or not the image area E has been set (ST84). When it is determined that the image area E has been set (ST84: YES), the processing illustrated in FIG. 19 ends.
[0093]
On the other hand, if it is determined that the image area E has not been set (ST84: NO), the image area setting unit 24 sets the image area E and the inattentive time of the area E (ST85). Then, the process illustrated in FIG. 19 ends.
[0094]
The image area E here is as shown in FIG. 20, and is the same as the person described with reference to FIG. When the position of the eye when viewing the left door mirror is determined, the upper end of the image area E can be determined as described above. For this reason, the image area setting unit 24 determines the upper end of the image area G based on the position of the eye when the left door mirror is viewed, and sets the inattentive time of the area G.
[0095]
Description will be made again with reference to FIG. If it is determined that either the image area C has not been set or the learning of the eye position when viewing the left door mirror has been completed (ST82: NO), the process proceeds to step ST86. I do. Then, the learning unit 23 determines whether or not the image area B has not been set and the learning of the eye position when the right door mirror is viewed has been completed (ST86).
[0096]
Then, when it is determined that the image area B has not been set and the learning of the eye position at the time of viewing the right door mirror has been completed (ST86: YES), the image area setting unit 24 sets the image area G, and further sets the image area G The inattentive time of G is set (ST87).
[0097]
The image area G here is as shown in FIG. 20 and is an area having a size obtained by adding the image area B and the image area D. The reason why the image area G is set in step ST87 is that the boundary between the front gaze area and the image area B can be set based on the position of the eye when the right door mirror is viewed. Also, the boundary between the image area B and the image area D cannot be set only by the position of the eye when the right door mirror is viewed.
[0098]
FIG. 19 is referred to again. After the setting of the image area G and the inattentive allowable time of the area G, the process proceeds to step ST84. Then, through the same processing as described above, the processing shown in FIG. 19 ends.
[0099]
If it is determined that either the image area B has not been set or the learning of the eye position at the time of viewing the right door mirror has been completed (ST86: NO), the process proceeds to step ST88. Move to Then, the learning unit 23 determines whether or not the image area D has not been set and the learning of the eye position at the time of the right rear view has been completed (ST88).
[0100]
Then, when it is determined that the image area D has not been set and the learning of the eye position at the time of the right rear view has been completed (ST88: YES), the image area setting unit 24 sets the image area D, and further sets the image area D The inattentive time of D is set (ST89).
[0101]
The image area D here is as shown in FIG. 20, and is the same as that described with reference to FIG. When the position of the eye at the time of the right rear view is determined, the boundary between the image area D and the image area B is determined as described above. Therefore, the image area setting unit 24 determines the image area D, and further looks aside at the area D. Set the allowable time. Then, the process illustrated in FIG. 19 ends.
[0102]
If it is determined that either the image area D has not been set or the learning of the eye position at the time of the right rear view has been completed is not satisfied (ST88: NO), the processing illustrated in FIG. 19 is performed. Ends.
[0103]
In this manner, according to the inattentiveness detection device 1 according to the present embodiment, a predetermined number of image areas having different inattentive allowable times based on the position of the eye on the captured image are set. Normally, the driver visually recognizes a relatively large amount of a rearview mirror or the like. Therefore, by using the position of the eye on the captured image as a reference, an image area can be set based on the position of a room mirror or the like. In addition, since it is based on the position of the rearview mirror or the like, a permissible inattentive time taking into account pillars or the like that block the driver's view can be allocated to each of the predetermined number of image areas.
[0104]
Then, inattention is detected based on the allowable inattentive time allocated to each image region and the position of the eye to be tracked. For this reason, inattentiveness detection can be performed based on the allowable inattentive time allocated to each image area in conformity with the in-vehicle environment.
[0105]
Therefore, it is possible to improve the accuracy of detecting the driver's inattentive behavior.
[0106]
The learning unit 23 learns the positions of the eyes on the captured image when viewing the positions of the left and right door mirrors and the room mirror. Usually, during driving, the driver visually recognizes many positions of the left and right door mirrors and the room mirror. For this reason, the positions of the detected eyes are large when looking at the left and right door mirrors and the room mirror, and the amount of data obtained by the detection is larger than when looking at other structures inside the vehicle. More. Therefore, the time required to determine the image area is shorter than when another structure inside the vehicle is viewed. Therefore, it is possible to shorten the time until the setting of the image area.
[0107]
Further, the learning section 23 learns the position of the eye on the captured image when the driver's eye reaches the tracking limit position. Usually, during driving, the driver checks the right rear relatively frequently. For this reason, the amount of data obtained by the detection is larger than when a structure inside another vehicle is viewed. Therefore, the time required to determine the image area is shorter than when another structure inside the vehicle is viewed. Therefore, it is possible to shorten the time until the setting of the image area.
[0108]
Further, the learning unit 23 learns the position of the eye on the captured image when the driver's eye reaches one of the left and right tracking limit positions. Then, based on the learned eye position, the position of the eye on the captured image when the driver's eye reaches the other tracking limit position is estimated. For this reason, even if the position of the eye on the captured image when the other tracking limit position is reached is not detected, the image area can be set based on the position obtained by estimation. Therefore, it is possible to shorten the time until the setting of the image area.
[0109]
Further, the image area setting unit 24 sequentially sets an image area every time the position of the eye on the captured image is learned. For this reason, the image area is set without waiting for the learning of the positions of all the eyes, so that it is possible to shorten the time until the inattentive detection is performed.
[0110]
In the above embodiment, the processing shown in FIG. 19 may be executed instead of steps ST19, ST20, and ST21 shown in FIG.
[0111]
Next, a second embodiment of the present invention will be described. The inattentiveness detection device 2 according to the second embodiment is similar to that of the first embodiment, but differs from that of the first embodiment in that an instructing unit (instruction means) 40 is provided.
[0112]
Hereinafter, an inattentiveness detection device 2 according to the second embodiment will be described. FIG. 21 is a configuration diagram of the inattentiveness detection device 2 according to the second embodiment of the present invention. The instructing unit 40 causes the learning unit 23 to learn the position of the eye when operated. Specifically, the instruction unit 40 is configured by a switch or the like installed below the front of the vehicle.
[0113]
FIG. 22 is a functional block diagram of the inattentiveness detection device 2 according to the second embodiment of the present invention. As shown at the same time, the instruction unit 40 is connected to the learning unit 23 in the processing device 20. The instruction section 40 sends a predetermined signal to the learning section 23 when operated. Then, the learning unit 23 receiving this starts learning the position of the eye.
[0114]
FIG. 23 is a flowchart showing a detailed operation of the processing device 20 shown in FIG. Steps ST90 to ST96 and steps ST100 to T103 in FIG. 13 are the same as steps ST10 to ST19 and steps ST20 to T23 shown in FIG.
[0115]
In step ST96, the learning section 23 performs a learning process of the position of the eye. At this time, when the driver operates the instruction unit 40, a predetermined signal is transmitted.
[0116]
The learning unit 23 determines whether or not the instruction unit 40 has been operated by determining whether or not a predetermined signal has been transmitted in step ST97. If it is determined that instruction unit 40 has not been operated (ST97: NO), the process returns to step ST90.
[0117]
On the other hand, when it is determined that the instruction unit 40 has been operated (ST97: YES), the learning unit 23 starts a learning process according to the instruction (ST98). The learning process performed at this time is the same as that shown in FIGS. 14 to 17, but differs from the process of step ST <b> 96 in that the position currently being visually recognized by the driver is forcibly learned. That is, if the instruction unit 40 is operated while the driver is viewing the rearview mirror, the position is forcibly learned.
[0118]
Then, the learning unit 23 determines whether or not learning has been completed for the learning process performed in response to the operation of the instruction unit 40 (ST99). If it is determined that learning has not been completed (ST99: NO), the process returns to step ST90. On the other hand, when it is determined that the learning has been completed (ST99: YES), image region setting section 24 determines whether or not an image region has been set (ST100). Thereafter, the same processing as in steps ST20 to T23 in FIG. 13 is performed.
[0119]
In this manner, according to the inattentiveness detection device 2 according to the present embodiment, similarly to the first embodiment, it is possible to improve the accuracy of the detection of the driver's inattentive act, and the time required to set the image area is improved. Can be shortened. Further, it is possible to shorten the time until the inattentive detection is performed.
[0120]
An instruction unit 40 is provided which causes the learning unit 23 to learn the position of the eye when operated. Therefore, the driver can easily learn the position of the eye only by operating the instruction unit 40, and the time until the setting of the image area is reduced. Therefore, it is possible to perform inattentiveness detection early from the start of operation or the like.
[0121]
Next, a third embodiment of the present invention will be described. The inattentiveness detecting device 3 according to the third embodiment is the same as that of the first embodiment, but includes a running state detecting unit (running state detecting unit) 50 and an allowable time changing unit (allowable time changing unit) 26. And is different from that of the first embodiment.
[0122]
Hereinafter, an inattentiveness detection device 3 according to the third embodiment will be described. FIG. 24 is a configuration diagram of the inattentiveness detection device 3 according to the third embodiment of the present invention. The traveling state detection unit 50 detects the traveling state of the vehicle, and includes a vehicle speed sensor 51, an inter-vehicle distance sensor 52 (distance sensor), and a navigation device 53.
[0123]
The vehicle speed sensor 51 detects a vehicle speed of the vehicle, and includes, for example, a sensor provided on an output side of the automatic transmission, a wheel speed sensor provided for detecting rotation of a wheel, and the like. The inter-vehicle distance sensor 52 detects the distance between the preceding vehicle and the host vehicle, and includes, for example, image processing using a CCD camera, an infrared camera, an ultrasonic sensor, a millimeter wave radar, a laser radar, and the like.
[0124]
The navigation device 53 guides the vehicle to the destination by presenting the driver with the vehicle position, the destination, the route to the destination, and the like. Further, the navigation device 53 has a function of presenting information on the type of road on the map to the driver.
[0125]
FIG. 25 is a functional block diagram of the inattentiveness detection device 3 according to the third embodiment of the present invention. As shown at the same time, the inattentiveness detecting device 3 according to the third embodiment has an allowable time changing unit 26 in the processing device 20.
[0126]
The permissible time changing unit 26 changes the inattentive permissible time according to an output signal from the traveling state detecting unit 50. Specifically, based on the vehicle speed signal from the vehicle speed sensor 51, the allowable time change unit 26 shortens the inattentive time as the speed of the vehicle increases. Further, the allowable time changing unit 26 shortens the inattentive time as the own vehicle approaches the preceding vehicle based on the distance signal from the following distance sensor 52. Further, the permissible time changing unit 26 determines, based on the information about the type of the road from the navigation device 53, the time when the vehicle is traveling on a general road, rather than the inattentive permissible time when the vehicle is traveling on the car dedicated road. Reduce inattention time.
[0127]
Next, an operation in a case where the inattentive-view allowable time is changed based on the vehicle speed will be described. FIG. 26 is a flowchart showing the detailed operation of the processing device 20 shown in FIG. 24, and shows an example in which the inattentive time is changed according to the vehicle speed. FIG. 26 illustrates a case where the allowable inattentive time of the image area A is changed.
[0128]
First, allowable time changing section 26 receives the vehicle speed signal from vehicle speed sensor 51 and detects the vehicle speed (ST110). Then, the allowable time changing unit 26 calculates a numerical value or the like corresponding to the vehicle speed. The allowable time changing unit 26 reads from the inattentive detection unit 25 information on the inattentive allowable time of the image area A set by the image area setting unit 24. Then, based on the information such as numerical values and the read information on the allowable inattentive time, the allowable inattentive time is changed and set (ST111).
[0129]
At the time of this change, the permissible time changing unit 26 changes the permissible inattentive time using a predetermined arithmetic expression, map data, or the like. FIG. 27 is an explanatory diagram illustrating an example of the inattentive viewing allowable time according to the vehicle speed. Specifically, the permissible time changing unit 26 shortens the inattentive permissible time as the vehicle speed increases, as shown in FIG. Note that the regions A to E in FIG. 27 indicate the image regions A to E described with reference to FIG.
[0130]
FIG. 26 is referred to again. After changing the inattentive-view allowable time, the process proceeds to step ST112, and ends after the process of step ST112 and thereafter. The processing of steps ST112 to ST117 is the same as the processing of steps ST70 to ST75 shown in FIG.
[0131]
Next, an operation in the case of changing the inattentive-view allowable time based on the inter-vehicle distance will be described. FIG. 28 is a flowchart showing the detailed operation of the processing device 20 shown in FIG. 24, and shows an example in which the inattentive viewing allowable time is changed according to the following distance. FIG. 26 illustrates a case where the allowable inattentive time of the image area A is changed.
[0132]
First, allowable time changing section 26 receives a distance signal from inter-vehicle distance sensor 52 and detects the distance to the preceding vehicle (ST120). Then, the allowable time changing unit 26 obtains a numerical value or the like corresponding to the inter-vehicle distance. The allowable time changing unit 26 reads from the inattentive detection unit 25 information on the inattentive allowable time of the image area A set by the image area setting unit 24. Then, based on the information such as the numerical values and the read information on the allowable inattentive time, the allowable inattentive time is changed and set (ST121).
[0133]
At the time of this change, the permissible time changing unit 26 changes the permissible inattentive time using a predetermined arithmetic expression, map data, or the like. FIG. 29 is an explanatory diagram showing an example of the inattentive viewing time according to the vehicle speed. Specifically, as shown in FIG. 29, the permissible time change unit 26 shortens the permissible inattentive time as the host vehicle approaches the preceding vehicle. Note that the regions A to E in FIG. 27 indicate the image regions A to E described with reference to FIG.
[0134]
FIG. 28 is referred to again. After the change of the inattentive-view allowable time, the process proceeds to step ST122, and ends after the process of step ST122 and thereafter. The processing of steps ST122 to ST127 is the same as the processing of steps ST70 to ST75 shown in FIG.
[0135]
Next, an operation in the case of changing the allowable inattentive time based on the information on the type of road will be described. FIG. 30 is a flowchart showing the detailed operation of the processing device 20 shown in FIG. 24, and shows an example in which the inattentive time is changed based on the road type information. FIG. 30 illustrates a case where the allowable inattentive time of the image area A is changed.
[0136]
First, the allowable time changing unit 26 inputs the road type information from the navigation device 53, and determines whether or not the own vehicle has entered the automobile exclusive road from the general road (ST130).
[0137]
When it is determined that the own vehicle is not on the automobile exclusive road from the general road (ST130: NO), the process proceeds to step ST132. On the other hand, when it is determined that the own vehicle has entered the automobile exclusive road from the general road (ST130: YES), the allowable time changing unit 26 sets the information of the inattentive time of the image area A set by the image area setting unit 24 as information. Read from the inattentiveness detection unit 25. Then, the read inattentive allowable time is set long. More specifically, the permissible time changing unit 26 sets “permissible inattentive time = permissible inattentive time × 1.2” and sets the new permissible inattentive time obtained (ST131).
[0138]
After that, the allowable time changing unit 26 inputs the road type information from the navigation device 53, and determines whether or not the own vehicle has entered the general road from the motorway (ST132).
[0139]
If it is determined that the vehicle is not on a general road from the motorway (ST132: NO), the process proceeds to step ST134. On the other hand, when it is determined that the own vehicle has entered the general road from the motorway (ST132: YES), the allowable time changing unit 26 sets the information of the inattentive time of the image area A set by the image area setting unit 24 as the information. Read from the inattentiveness detection unit 25. Then, the read inattentive time is set to be short. More specifically, the permissible time changing unit 26 sets “acceptable inattentive time = permissible inattentive time × 0.83”, and sets the obtained new permissible inattentive time (ST133).
[0140]
After the change of the inattentive-view allowable time, the process proceeds to step ST134, and ends after the process of step ST134 and thereafter. The processing of steps ST134 to ST139 is the same as the processing of steps ST70 to ST75 shown in FIG.
[0141]
In this way, according to the inattentiveness detection device 3 according to the present embodiment, similarly to the first embodiment, it is possible to improve the accuracy of the detection of the driver's inattentive act, and to reduce the time required for setting the image area. Can be shortened. Further, it is possible to shorten the time until the inattentive detection is performed.
[0142]
In addition, the traveling state of the vehicle is detected, and the allowable inattentive time is changed according to the detection result. For this reason, the permissible inattentive time can be changed in consideration of not only the vehicle interior environment but also the environment around the vehicle. Therefore, it is possible to further improve the accuracy of detecting the driver's inattentive behavior.
[0143]
In addition, based on the vehicle speed signal from the vehicle speed sensor 51, the allowable inattentive time is shortened as the speed of the vehicle increases. For this reason, for example, when the vehicle speed is high, even if the driver is visually recognizing the vicinity of the front gaze area, the allowable inattentive time can be shortened, and the inattentive detection can be performed more appropriately.
[0144]
Further, based on the distance signal from the inter-vehicle distance sensor 52, the inattention time is reduced as the host vehicle approaches the preceding vehicle. For this reason, for example, when the distance to the preceding vehicle is short, the inattentive allowable time can be shortened even if the driver is visually recognizing the vicinity of the forward gaze area, and the inattentive detection can be performed more appropriately. it can.
[0145]
Generally, ordinary roads have pedestrians, traffic lights, and the like, and motorways do not have pedestrians, traffic lights, and the like. As described above, it is a fact that general roads have more opportunities to stop vehicles than motorways. In addition, roads for motorways are often wider. For this reason, the inattentiveness detection can be performed more appropriately by shortening the allowable inattentive time when traveling on a general road than when traveling on a motorway.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of an inattentiveness detection device according to a first embodiment of the present invention.
FIG. 2 is a functional block diagram of the inattentiveness detection device according to the first embodiment of the present invention.
FIG. 3 is an explanatory diagram of an initial process performed when a position detection unit detects a position of an eye.
FIG. 4 is an explanatory diagram illustrating a state when a position detection unit extracts a predetermined pixel.
FIG. 5 is an explanatory diagram illustrating a state in which pixels adjacent in the horizontal direction of an image are grouped.
FIG. 6 is an explanatory diagram illustrating a state after zoning by a position detection unit.
FIGS. 7A and 7B are explanatory diagrams showing how an eye position is tracked by a tracking unit. FIG. 7A shows an initial tracking area, and FIG. 7B shows a state of an eye detected after setting the initial tracking area. (C) shows the tracking area set based on the detected eye position, and (d) shows the eye position detected from the newly set tracking area.
FIG. 8 is an explanatory diagram showing a state when the driver visually recognizes the front, the rear-view mirror, and the left and right door mirrors, and FIG. 8A illustrates the position of the driver's eyes when the driver is visually recognizing the front; (B) shows the position of the driver's eye when looking at the rearview mirror, (c) shows the driver's eye position when looking at the right door mirror, and (d) shows It shows the position of the driver's eye when looking at the left door mirror, and (e) shows the coordinate position of the eye according to the driver's view point.
FIG. 9 is an explanatory diagram illustrating a relationship between a dwell time of a driver's eye and a vehicle speed when a mirror is viewed.
FIGS. 10A and 10B are explanatory diagrams illustrating examples of a predetermined number of image areas having different allowable inattentive times, in which FIG. 10A illustrates an example of a predetermined number of image areas, and FIG. Is shown.
11A and 11B are explanatory diagrams of a method of setting an image area by an image area setting unit, wherein FIG. 11A illustrates a state of a face direction when a driver looks at a left door mirror, and FIG. Shows the face orientation when looking at the right door mirror.
FIG. 12 is an explanatory diagram of a specific example of an image area set on a captured image.
FIG. 13 is a flowchart showing a detailed operation of the processing device shown in FIG. 1;
FIG. 14 is a flowchart showing details of the eye position learning process (ST19) shown in FIG. 13, and shows a learning process when the room mirror is visually recognized.
15 is a flowchart showing details of an eye position learning process (ST19) shown in FIG. 13 and shows a learning process when the left door mirror is visually recognized.
FIG. 16 is a flowchart showing details of an eye position learning process (ST19) shown in FIG. 13, and shows a learning process when the right door mirror is viewed.
FIG. 17 is a flowchart showing details of an eye position learning process (ST19) shown in FIG. 13, and shows a learning process at the time of right rearward visual recognition.
FIG. 18 is a detailed flowchart of an inattentive judgment (ST22) shown in FIG.
FIG. 19 is an explanatory diagram illustrating a process in a case where an image area is set in stages.
FIG. 20 is an explanatory diagram illustrating an example of an image area set in a stepwise manner.
FIG. 21 is a configuration diagram of an inattentiveness detection device according to a second embodiment of the present invention.
FIG. 22 is a functional block diagram of an inattentiveness detection device according to a second embodiment of the present invention.
23 is a flowchart showing a detailed operation of the processing device shown in FIG. 21.
FIG. 24 is a configuration diagram of an inattentiveness detection device according to a third embodiment of the present invention.
FIG. 25 is a functional block diagram of an inattentiveness detection device according to a third embodiment of the present invention.
26 is a flowchart showing a detailed operation of the processing device shown in FIG. 24, and shows an example in a case where an inattentive viewing time is changed according to a vehicle speed.
FIG. 27 is an explanatory diagram showing an example of an inattentive viewing allowable time according to a vehicle speed.
FIG. 28 is a flowchart showing a detailed operation of the processing device shown in FIG. 24, and shows an example of a case where an allowable inattentive time is changed according to an inter-vehicle distance.
FIG. 29 is an explanatory diagram showing an example of an inattentive viewing time according to a vehicle speed.
FIG. 30 is a flowchart showing a detailed operation of the processing device shown in FIG. 24, and shows an example in a case where an allowable inattentive time is changed based on road type information.
[Explanation of symbols]
1-3 ... inattentive detection device
10 ... Imaging unit (imaging means)
20 Processing unit
21 ... Position detector
22: tracking unit (tracking means)
23 Learning unit (learning means)
24. Image area setting unit (image area setting means)
25 ... inattentiveness detection unit (inattentiveness detection means)
26 ... Allowable time changing unit (Allowable time changing means)
30 ... Buzzer
40 ... instruction unit (instruction means)
50: running state detecting section (running state detecting means)
51… Vehicle speed sensor
52: Inter-vehicle distance sensor
53 ... Navigation device
AG: Image area

Claims (11)

Photographing means for photographing the face of the driver of the vehicle;
A tracking unit that tracks the position of the driver's eyes based on a captured image including the driver's face obtained by the imaging unit;
Learning means for determining the movement amount and the dwell time of the eye position output in time series from the tracking means, and learning the position of the eye on the captured image when the driver looks at the predetermined position,
Image area setting means for setting a predetermined number of image areas having different inattentive viewing times based on the position of the eye on the captured image learned by the learning means,
An inattentiveness detection unit that performs inattentiveness detection based on the inattentive allowance time allocated to the image area set by the image area setting unit,
An inattentiveness detection device comprising: The inattentiveness detection apparatus according to claim 1, further comprising an instruction unit that, when operated, causes the learning unit to learn an eye position. Traveling state detection means for detecting the traveling state of the vehicle,
According to an output signal from the traveling state detecting means, an allowable time changing means for changing an inattentive allowable time,
The inattentiveness detection apparatus according to claim 1, further comprising: The traveling state detecting means includes a vehicle speed sensor that detects a vehicle speed of the vehicle,
The inattentiveness detection device according to claim 3, wherein the allowable time change unit decreases the inattentive allowable time as the vehicle speed increases based on a vehicle speed signal from a vehicle speed sensor. The running state detecting means includes a distance sensor that detects a distance between the preceding vehicle and the own vehicle,
5. The inattentive side according to claim 3, wherein the allowable time changing unit shortens the inattentive side allowable time as the host vehicle approaches the preceding vehicle based on a distance signal from a distance sensor. 6. Detection device. The driving state detection unit includes a navigation device that presents information about a type of a road on a map to a driver,
The permissible time changing means is based on the information on the type of the road from the navigation device, and the permissible time when the vehicle is traveling on a general road, rather than the permissible time when the vehicle is traveling on a dedicated road. The inattentiveness detection apparatus according to any one of claims 3 to 5, wherein the distance is shortened. 7. The learning device according to claim 1, wherein the learning unit learns, as the predetermined position, an eye position on a captured image when viewing at least one of a left and right door mirror and a room mirror. 8. An inattentiveness detection device according to any one of the preceding claims. The said learning means learns the position of the eye on the captured image when the driver's eye reaches the tracking limit position, assuming that the driver looks at the predetermined position. Item 7. The inattentiveness detection device according to any one of items 7. The learning means learns the position of the eye on the captured image when the driver's eye reaches one of the left and right tracking limit positions, assuming that the driver looks at the predetermined position, and The method according to claim 1, further comprising: estimating a position of the eye on the captured image when the driver's eye reaches the other tracking limit position based on the position. Inattentive detector. The inattentive look according to any one of claims 1 to 9, wherein the image area setting unit sequentially sets an image area each time a position of an eye on a captured image is learned. Detection device. The position of the driver's eye is tracked based on the captured image including the driver's face obtained by photographing the face of the vehicle driver, and the movement amount and the stay of the driver's eye position specified by the tracking are tracked. The time is determined, the position of the eye on the captured image when the driver looks at the predetermined position is learned, and a predetermined number of images having different inattentive times with respect to the eye position on the learned captured image are determined. An inattentiveness detection device that sets an area and performs inattentiveness detection based on the allowable inattentive time of the image area.

Images