JP2010066968A - Visual line state detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a visual line state detection device capable of verifying the validity of detection result of a visual line of driver. <P>SOLUTION: The visual line state detection device 10 for detecting the visual line of the driver in his/her own vehicle comprises a driver monitor sensor 1 and a direction detection part 2 for detecting the visual line of the driver L, a steering angle sensor 3 for detecting a steering angle R of the own vehicle, and a wrong detection determination part 4 for determining whether or not the detection result of the visual line of the driver L detected by the driver monitor sensor 1 and the direction detection part 2 is proper based on the steering angle R detected by the steering angle sensor 3. Whether or not the detection result of the visual line of the driver L detected by the driver monitor sensor 1 and the direction detection part 2 is proper can be determined by the wrong detection determination part 4 based on the steering angle R detected by the steering angle sensor 3. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a gaze state detection device that detects a gaze direction of a driver of a host vehicle.

Conventionally, a technique for determining whether or not the driver of the own vehicle is looking aside has been proposed. For example, Patent Literature 1 detects a driver's visual direction, detects a visual recognition area that is subject to visual recognition by the driver based on the visual direction, and sets a risk value corresponding to the visual recognition area. A technique for calculating the degree of risk based on detection, that is, looking aside is disclosed.
JP 2004-114931 A

  In the technique described in Patent Document 1, a driver's face is captured using a camera to obtain a face image, and a driver's eye movement amount and eyelid opening / closing amount are calculated based on the face image. Based on the result of this calculation, the driver's viewing direction is detected.

  However, if the driver is wearing eyeglasses, a face is wearing an ornament or a mask, or a disturbance occurs in the visual direction detection device, the driver's gaze direction and other characteristic points may be mistaken. There is a risk of detection. For this reason, there exists a possibility that the precision of the detection result of the gaze direction based on the feature point of a driver | operator's face may worsen.

  Then, an object of this invention is to provide the gaze state detection apparatus which can verify the validity of the detection result of a driver | operator's gaze direction.

  That is, the gaze state detection device according to the present invention is a gaze state detection device that detects the gaze direction of the driver of the host vehicle, the direction detection unit that detects the gaze direction of the driver, and the steering angle of the host vehicle. Steering angle detection means for detecting, and determination means for determining whether the detection result of the line-of-sight direction detected by the direction detection means is valid based on the steering angle detected by the steering angle detection means. It is characterized by that.

  The driver of the vehicle has a tendency to visually recognize the tip of the traveling direction of the own vehicle. Therefore, the driver's line-of-sight direction and the traveling direction of the host vehicle corresponding to the steering angle of the host vehicle usually coincide. Therefore, it can be determined by the determination means based on the steering angle detected by the steering angle detection means whether the detection result of the line-of-sight direction detected by the direction detection means is valid. Thereby, the validity of the detection result of a driver | operator's gaze direction can be verified.

  In addition, the direction detection means preferably detects the line-of-sight direction by acquiring the driver's face orientation angle and pupil position.

  As a result, the driver's face orientation angle and pupil position are acquired by the direction detection means, and based on this, the line-of-sight direction is detected by the direction detection means. In this way, since the line-of-sight direction is detected based on the acquired face orientation angle and pupil position, the accuracy of the detection result of the line-of-sight direction detected by the direction detection unit can be increased.

  In addition, the determination unit determines the direction of the line of sight when the detection result of the line of sight detected by the direction detection unit does not match the detection result of the traveling direction of the host vehicle based on the steering angle detected by the steering angle detection unit. It is also preferable to determine that the detection result is not valid.

  As a result, when the detection result of the gaze direction detected by the direction detection unit and the detection result of the traveling direction of the host vehicle based on the steering angle detected by the steering angle detection unit do not match, the detection result of the gaze direction Is determined to be invalid by the determining means. Thus, using the tendency of the driver of the vehicle to visually recognize the tip of the traveling direction of the host vehicle, the traveling direction of the host vehicle based on the detection result of the line-of-sight direction detected by the direction detecting unit and the steering angle. When the detection result does not coincide with the detection result, it is determined that the detection result in the gaze direction is not valid. Therefore, the validity of the detection result in the gaze direction can be verified with high accuracy.

  ADVANTAGE OF THE INVENTION According to this invention, the gaze state detection apparatus which can verify the validity of the detection result of a driver | operator's gaze direction can be provided.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings. In order to facilitate understanding of the description, the same components in the drawings are denoted by the same reference numerals as much as possible, and redundant description will be omitted.

  First, the configuration of the visual line state detection device according to the embodiment of the present invention will be described with reference to FIG. FIG. 1 is a schematic configuration diagram of a gaze state detection apparatus 10 according to the present embodiment. The line-of-sight state detection device 10 is attached to a vehicle such as an automobile (hereinafter referred to as the host vehicle), detects the direction of presence of the visual recognition area (that is, the line-of-sight direction) visually recognized by the driver of the host vehicle, and the detection result is It is a device that determines whether or not it is appropriate based on the steering state of the host vehicle (that is, the direction of rotation of the steering wheel, etc.).

  The line-of-sight state detection apparatus 10 according to the present embodiment includes a driver monitor sensor 1 (direction detection unit), a direction detection unit 2 (direction detection unit), a steering angle sensor 3 (steering angle detection unit), and an erroneous detection determination unit. 4 (determination means). In addition, the function by the direction detection part 2 and the false detection determination part 4 is implement | achieved by ECU (not shown) which is an electronic control apparatus mounted in the inside of the own vehicle, for example. The ECU is a unit whose main component is a microcomputer including a CPU, a ROM, a RAM, and the like.

  The driver monitor sensor 1 is a monitoring sensor that is attached to the steering column and captures the face of the driver of the host vehicle and acquires a face image (hereinafter referred to as a face image). The driver monitor sensor 1 is configured by a fisheye camera using a semiconductor element such as a CCD or a CMOS, for example. The face image data acquired by the driver monitor sensor 1 is transmitted to the direction detection unit 2.

  The direction detection unit 2 performs processing based on the face image acquired by the driver monitor sensor 1 (for example, image processing), and an angle at which the driver's face (head) faces the front of the host vehicle, so-called face orientation angle. It is the part which detects and acquires. In addition, the direction detection unit 2 performs processing (for example, image processing) based on the face image acquired by the driver monitor sensor 1, and detects and acquires the driver's pupil position (that is, pupil position) with reference to the face. . And the direction detection part 2 detects a driver | operator's gaze direction based on this acquired face direction angle and pupil position. Data of the line-of-sight direction detected by the direction detection unit 2 is transmitted to the erroneous detection determination unit 4.

  The steering angle sensor 3 is a sensor that detects the steering angle of the host vehicle, that is, the rotation direction and rotation angle of the steering wheel. The steering angle data detected by the steering angle sensor 3 is transmitted to the erroneous detection determination unit 4.

  The erroneous detection determination unit 4 determines whether or not the detection result of the line-of-sight direction detected by the direction detection unit 2 is valid, based on the state of the steering angle detected by the steering angle sensor 3 (that is, the rotation direction of the steering wheel or zero). This is a portion to be determined based on the time required for returning to the neutral position. First, the erroneous detection determination unit 4 determines whether or not a line-of-sight direction of a certain angle or more is held for a certain period of time with reference to the front of the host vehicle. It is determined whether or not the result is valid (that is, whether or not it is erroneously detected).

  Here, the details of this determination method will be described. The driver of the host vehicle tries to visually recognize the tip of the traveling direction of the host vehicle by directing a face or line of sight toward the tip of the traveling direction of the host vehicle during steering while rotating the steering wheel, for example, when turning right or left. Have a tendency to Therefore, the direction in which the visual recognition area visually recognized by the driver (that is, the line-of-sight direction) and the traveling direction of the own vehicle corresponding to the steering angle of the own vehicle (that is, the steering direction) usually coincide with each other within a predetermined error range. . Therefore, whether or not the detection result of the line-of-sight direction detected by the direction detection unit 2 is valid can be determined based on the steering angle detected by the steering angle sensor 3. Thereby, it becomes possible to prevent erroneous detection of the line-of-sight direction.

  The details of the validity determination method for the detection result of the direction in which the visual recognition area exists (that is, the line-of-sight direction) will be described with reference to FIGS. Each of FIG. 2 and FIG. 3 is an explanatory diagram for explaining the details of the validity determination method of the detection result of the direction in which the visual recognition area exists. Each of FIGS. 2 and 3 shows the line-of-sight direction of the driver D detected by the direction detection unit 2 and the steering angle of the steering wheel W detected by the steering angle sensor 3 on the paper surface. 2 and 3, the vertical direction on the paper surface indicates the front-rear direction of the host vehicle (the upward direction on the paper surface is the front direction of the host vehicle), and the left-right direction on the paper surface indicates the vehicle width direction (right side on the paper surface). The direction indicates the right side of the vehicle.

  In FIG. 2A, the direction in which the face corresponding to the acquired driver D's face direction angle A is the diagonally forward right direction of the host vehicle, and further, the acquired pupil position of the driver D is the driving position. It is shown that it is substantially centered when viewed from the person D and the line-of-sight direction L faces the front direction substantially perpendicular to the face. As a result, the detection result of the line-of-sight direction L is a direction diagonally forward to the right of the host vehicle. In addition, the rotation direction corresponding to the detected steering angle R of the steering wheel W is the right rotation direction (that is, the traveling direction of the host vehicle is the diagonally forward right direction).

  In this case, the erroneous detection determination unit 4 detects the detection result of the line-of-sight direction L (that is, the direction diagonally forward right of the host vehicle) and the detection result of the traveling direction of the host vehicle based on the steering angle R (that is, the progress of the host vehicle). The direction of the right diagonally front direction) coincides within a predetermined error range, so that the detection result of the direction in which the visual recognition area exists is determined to be valid.

  In FIG. 2 (b), the direction in which the face corresponding to the acquired driver D's face angle A is the diagonally forward left direction of the host vehicle, and the acquired pupil position of the driver D is the driving position. It shows that it is the right end when viewed from the person D and the line-of-sight direction L faces the right direction substantially parallel to the face. As a result, the detection result of the line-of-sight direction L is a direction diagonally forward to the right of the host vehicle. In addition, the rotation direction corresponding to the detected steering angle R of the steering wheel W is the right rotation direction (that is, the traveling direction of the host vehicle is the diagonally forward right direction).

  In this case, the erroneous detection determination unit 4 detects the detection result of the line-of-sight direction L (that is, the direction diagonally forward right of the host vehicle) and the detection result of the traveling direction of the host vehicle based on the steering angle R (that is, the progress of the host vehicle). The direction of the right diagonally front direction) coincides within a predetermined error range, so that the detection result of the direction in which the visual recognition area exists is determined to be valid.

  In FIG. 3A, the direction of the face corresponding to the acquired driver D's face orientation angle A is the direction diagonally forward to the right of the host vehicle, and the acquired pupil position of the driver D is the driving position. It is shown that it is substantially centered when viewed from the person D and the line-of-sight direction L faces the front direction substantially perpendicular to the face. As a result, the detection result of the line-of-sight direction L is a direction diagonally forward to the right of the host vehicle. In addition, the rotation direction corresponding to the detected steering angle R of the steering wheel W is the left rotation direction (that is, the traveling direction of the host vehicle is a diagonally forward left direction).

  In this case, the erroneous detection determination unit 4 detects the detection result of the line-of-sight direction L (that is, the direction diagonally forward right of the host vehicle) and the detection result of the traveling direction of the host vehicle based on the steering angle R (that is, the progress of the host vehicle). The direction of the diagonally left front direction does not match even when a predetermined error range is taken into consideration, and therefore the detection result of the direction in which the visual recognition area exists is determined to be invalid (that is, erroneous detection).

  In FIG. 3B, the direction in which the face corresponding to the acquired driver D's face orientation angle A is the diagonally forward left direction of the host vehicle, and the acquired pupil position of the driver D is the driving position. It shows that it is the right end when viewed from the person D and the line-of-sight direction L faces the right direction substantially parallel to the face. As a result, the detection result of the line-of-sight direction L is a direction diagonally forward to the right of the host vehicle. In addition, the rotation direction corresponding to the detected steering angle R of the steering wheel W is the left rotation direction (that is, the traveling direction of the host vehicle is a diagonally forward left direction).

  In this case, the erroneous detection determination unit 4 detects the detection result of the line-of-sight direction L (that is, the direction diagonally forward right of the host vehicle) and the detection result of the traveling direction of the host vehicle based on the steering angle R (that is, the progress of the host vehicle). The direction of the diagonally left front direction does not match even when a predetermined error range is taken into consideration, and therefore the detection result of the direction in which the visual recognition area exists is determined to be invalid (that is, erroneous detection).

  Subsequently, referring to FIG. 4, a series of flow from the imaging process of the driver's face to the determination process of the validity of the detection result of the presence direction of the visual recognition area (that is, the visual line direction), which is executed by the visual line state detection device 10. Will be explained. FIG. 4 is a flowchart for explaining a series of flows of these processes. The processing shown in the flowchart of FIG. 4 is mainly performed by the driver monitor sensor 1, the steering angle sensor 3, and the ECU (which realizes the function of the direction detection unit 2 and the erroneous detection determination unit 4). It is repeatedly executed at a predetermined timing from when the power of the device 10 is turned on until it is turned off.

  First, the driver monitor sensor 1 captures the face of the driver of the own vehicle and acquires a face image (step S01). Then, the direction detection unit 2 performs processing (for example, image processing) based on the face image acquired by the driver monitor sensor 1, acquires the face orientation angle and the pupil position, and detects the gaze direction based on this (step) S02). Then, the steering angle sensor 3 detects the steering angle of the host vehicle (step S03).

  Next, the erroneous detection determination unit 4 determines whether the detection result of the line-of-sight direction detected by the direction detection unit 2 is appropriate based on the steering angle detected by the steering angle sensor 3 (step S04). ).

  Here, when the detection result of the gaze direction matches the detection result of the traveling direction of the host vehicle based on the steering angle within a predetermined error range, the gaze direction detection result is determined to be valid, and the series The process flow ends.

  On the other hand, if the detection result of the gaze direction and the detection result of the traveling direction of the host vehicle based on the steering angle do not match even when a predetermined error range is considered, the detection result of the gaze direction is not valid (i.e. Is determined) (step S05). Then, the series of processing flow ends.

  Continuously, the effect of this embodiment is demonstrated. According to this embodiment, it is determined based on the detected steering angle whether the detection result of the line-of-sight direction L is valid. Thereby, the validity of the detection result of the driver's line-of-sight direction L can be verified. That is, it can be determined whether or not the line-of-sight direction L is erroneously detected.

  Further, the detection result of the line-of-sight direction L and the detection result of the traveling direction of the host vehicle based on the steering angle R are predetermined using a tendency that the driver of the vehicle tries to visually recognize the tip of the traveling direction of the host vehicle. If the error range does not match even if the error range is considered, it is determined that the detection result of the line-of-sight direction L is not valid. For this reason, the validity of the detection result of the line-of-sight direction L can be verified with high accuracy.

  The preferred embodiment of the present invention has been described above, but the present invention is not limited to the above embodiment. For example, in the above embodiment, the direction detection unit 2 performs a face orientation angle acquisition process and a pupil position acquisition process. However, instead of the direction detection unit 2, the direction detection unit 2 performs a face direction angle acquisition process. It is good also as a structure divided into a part and a part which performs the acquisition process of a pupil position.

  In the above embodiment, the erroneous detection determination unit 4 is configured to determine the validity of the detection result of the line-of-sight direction L based on the detected steering angle state. The determination may be made based on information collected in the navigation system and infrastructure information such as road information obtained by an optical beacon antenna mounted on the vehicle. Thus, the accuracy of this determination can be further increased by increasing the information used for the determination.

1 is a schematic configuration diagram of a gaze state detection device according to the present embodiment. It is explanatory drawing for demonstrating the detail of the validity determination method of the detection result of a gaze direction. It is explanatory drawing for demonstrating the detail of the validity determination method of the detection result of a gaze direction. It is a flowchart for demonstrating a series of flows of the process performed with a gaze state detection apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Driver monitor sensor, 2 ... Direction detection part, 3 ... Steering angle sensor, 4 ... False detection determination part, 10 ... Gaze state detection apparatus, A ... Angle, D ... Driver, L ... Gaze direction, R ... Steering angle , W ... Steering wheel.

Claims (3)

A gaze state detection device for detecting a gaze direction of a driver of the own vehicle,
Direction detection means for detecting the driver's line-of-sight direction;
Steering angle detection means for detecting the steering angle of the host vehicle;
Determination means for determining whether or not the detection result of the line-of-sight direction detected by the direction detection means is valid based on the steering angle detected by the steering angle detection means;
A line-of-sight state detection apparatus comprising:   The gaze state detection apparatus according to claim 1, wherein the direction detection unit detects the gaze direction by acquiring a face direction angle and a pupil position of the driver. In the case where the detection result of the line-of-sight direction detected by the direction detection unit does not match the detection result of the traveling direction of the host vehicle based on the steering angle detected by the steering angle detection unit. The gaze state detection apparatus according to claim 1, wherein the gaze direction detection result is determined to be invalid.

Images