WO2014148025A1

WO2014148025A1 - Travel control device

Info

Publication number: WO2014148025A1
Application number: PCT/JP2014/001485
Authority: WO
Inventors: 泰斗渡邉
Original assignee: 株式会社デンソー
Priority date: 2013-03-21
Filing date: 2014-03-17
Publication date: 2014-09-25
Also published as: JP2014181020A

Abstract

A travel control device provided with: a control unit (170) that performs an automatic driving control for a vehicle on the basis of types of information necessary to travel; a facial-image acquiring means (160) that acquires a facial image of a driver driving the vehicle; and a evaluation unit (170, S110) that evaluates, in response to changes in facial orientations over time in the facial images, whether the driver is performing cautious movements. When cautious movements are not perceived within a prescribed time, the control unit (170) activates a warning unit (180) that warns the driver and changes the operation content of and/or disables the operation of a drive unit that is involved in changes to the travelling state of the automatic driving control. Thus, cautious movements by the driver can be clearly detected and measures for safe travel operation can be performed.

Description

Travel control device

Cross-reference of related applications

This disclosure is based on Japanese Application No. 2013-58728 filed on March 21, 2013, the contents of which are incorporated herein.

The present disclosure relates to a traveling control device that performs measures for safe traveling based on the driver's attention behavior during driving.

As a conventional travel control device, for example, the one shown in Patent Document 1 is known. The travel control device disclosed in Patent Document 1 includes a driver attention level setting unit that sets a degree of attention to be paid by a driver of the vehicle according to a surrounding state of the vehicle and a driving state of the vehicle, The driver attention level detection means that detects the degree of attention paid to the situation and the driving state, and the driver attention level setting means sets the degree of attention paid by the driver detected by the driver attention level detection means. At least one of the following actions: alerting the driver, decelerating the car, prohibiting the acceleration of the car, changing the direction of the car, and canceling the lane change of the car Traveling control means to perform.

In Patent Document 1, the driver attention level detection means detects the degree of attention given by the driver based on the driver's line-of-sight direction detected by the line-of-sight detection device. The degree of attention paid by the driver is the number of times that the driver has confirmed safety by looking at the preceding vehicle, pedestrian, roadside object, etc., or the interval at which safety confirmation has been performed. For example, when the driver performs a lane change, the driver's gaze and safety confirmation operation on the side mirror (or side or front) is periodically performed (for example, 200 ms or more per second). If not detected by the detection means, the actuator controller operates the alarm actuator to warn the driver and stop the lane change.

JP 2009-18625 A

In Patent Document 1, the degree of attention paid by the driver as described above is detected based on the driver's line-of-sight direction. The movement of a person's line of sight grasps the movement of the pupil of the eye, but the range of movement of the pupil is a limited small range in the open eye.

An object of the present disclosure is to provide a travel control device that can clearly detect a driver's attention behavior and take measures for safe driving.

According to the first aspect of the present disclosure, the travel control device includes a control unit that performs automatic driving control of the vehicle based on various types of information required for traveling,
Facial image acquisition means for acquiring a facial image of a driver driving the vehicle;
A determination unit configured to determine presence or absence of the driver's attention behavior in accordance with a time-series change in face orientation in the face image.

The control unit performs at least the operation of the warning unit for warning to the driver and the prohibition of the operation of the driving unit or the change of the operation related to the change of the driving state in the automatic driving control when the caution action is not recognized for a predetermined time. Run one.

According to the above configuration, the determination unit determines the presence / absence of the driver's attention behavior during the automatic driving control from the time-series change in face orientation in the face image. Compared with the case where the determination is made based on a small change in the line-of-sight direction, the change in the face direction can be greatly obtained, and the determination accuracy can be significantly improved. The control unit is configured to operate at least one of the operation of the warning unit for warning to the driver and the prohibition of the operation of the driving unit or the change of the operation related to the change of the driving state in the automatic driving control when the caution action is not recognized for a predetermined time. Therefore, safe driving can be realized with certainty.

According to the second aspect of the present disclosure, the travel control device that performs the automatic driving control of the vehicle includes:
A face image acquisition unit for acquiring driver's face image data;
Based on the acquired face image data, an attention behavior determination unit that determines the presence or absence of the driver's attention behavior according to a change state of the driver's time-series facial orientation;
When the attention behavior determination unit determines that there is no driver's attention behavior, at least one of an accelerator opening control device, a steering angle control device, and a brake control device that controls the operation of the warning unit and automatic driving An operation control unit that performs at least one of the operation control including prohibition or change of the operation related to the change of the traveling state.

This effect can provide the same effect as described above.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings.
It is a block diagram which shows the whole traveling control apparatus in 1st Embodiment. It is a flowchart which shows the safe driving | running | working driving | operation measure control which the control part in 1st Embodiment performs. It is a block diagram which shows the whole traveling control apparatus in 2nd Embodiment. It is a flowchart which shows the safe driving | running | working driving | operation measure control which the control part in 2nd Embodiment performs. It is a flowchart which shows the safe driving | running | working driving | operation measure control which the control part in 3rd Embodiment performs. It is a flowchart which shows the safe driving | running | working driving | operation measure control which the control part in 4th Embodiment performs.

Hereinafter, a plurality of modes for carrying out the present disclosure will be described with reference to the drawings. In each embodiment, parts corresponding to the matters described in the preceding embodiment may be denoted by the same reference numerals, and redundant description may be omitted. When only a part of the configuration is described in each mode, the other modes described above can be applied to the other parts of the configuration. Not only combinations of parts that clearly indicate that the combination is possible in each embodiment, but also a combination of the embodiments even if they are not clearly specified unless there is a problem with the combination. It is also possible.

(First embodiment)
A travel control device 100 according to the first embodiment will be described with reference to FIGS. 1 and 2. The travel control device 100 performs automatic driving control of the vehicle based on various information (described later) required for the vehicle to travel, monitors the driver's attention behavior during driving, It is a device that controls measures for safe driving when it is not allowed.

As shown in FIG. 1, the travel control apparatus 100 includes positioning information detection means 110, travel route information detection means 120, surrounding vehicle information detection means 130, surrounding environment recognition means 140, own vehicle information collection means 150, face image acquisition means. 160, a control unit 170, and the like. The control unit 170 is configured to give instructions to the accelerator opening control device 210, the steering angle control device 220, and the brake control device 230.

The positioning information detection means 110 is constituted by, for example, a GPS receiver (which may be dedicated to the system or may be provided for a car navigation device), and in real time based on the received GPS signal. This is means for detecting positioning information (longitude and latitude information) indicating the current position of the host vehicle. The positioning information detection unit 110 outputs the detected positioning information to the control unit 170 described later. The positioning information detection means 110 is capable of highly accurate positioning such as RTK (realtime kinetic) -GPS using an electronic reference point and D (differential) -GPS using correction data from a reference station. Use.

The travel route information detection unit 120 performs road-to-vehicle communication using radio waves or the like as a medium with a road machine (corresponding to an information providing unit) installed at a predetermined interval at a position along the travel lane of the vehicle. It is a detection means for receiving travel route information transmitted from the roadside. The travel route information detection means 120 outputs the detected travel route information to the control unit 170 described later. In this case, the travel route information transmitted from the roadside machine side includes, for example, the following information (a) to (d).

(A) Among a large number of reference points set at predetermined intervals on a travel lane provided during the automatic driving control of the vehicle, a plurality of reference points located in the traveling direction of the vehicle (that is, the vehicle should proceed next) High-precision location information (for example, longitude and latitude information)
(B) Congestion information relating to the destination of the vehicle on the road where the driving lane is set,
(C) Traffic signal information indicating the status of traffic signals existing at the destination of the vehicle on the road where the travel lane is set,
(D) Speed limit information and traffic regulation information about the road on which the travel lane is set.

The surrounding vehicle information detection means 130 acquires the current position information of the other vehicle by performing two-way communication with another vehicle located within a predetermined distance from the own vehicle, and This is a detection means having a function of transmitting current position information to another vehicle. The surrounding vehicle information detection means 130 outputs the surrounding vehicle information including the acquired current position information of other vehicles to the control unit 170 described later.

The surrounding environment recognition unit 140 can detect obstacles by a self-supporting method (or a self-supporting method combining them) using a CCD (Charge Coupled Device) camera, a millimeter wave or laser radar, an ultrasonic sensor, or the like. The recognition means is configured to be detected. The surrounding environment recognition unit 140 recognizes the environment around the host vehicle, the road surface condition, etc., recognizes the presence / absence and location of obstacles on the road, and outputs the recognition information to the control unit 170 described later as surrounding environment information. It is like that.

The own vehicle information collecting means 150 is a means for collecting information on the own vehicle, that is, vehicle speed, steering angle, engine speed, and other diagnostic information, and outputs the collected own vehicle information to the control unit 170 described later. It has become.

The face image acquisition means 160 acquires a driver's face image during driving, and is mounted on the steering column, for example, facing the driver's face in the passenger compartment. As the face image acquisition means 160, for example, a CCD camera or the like can be used. The face image acquisition unit 160 acquires a driver's face during driving in a time series in a predetermined cycle, and outputs the acquired face image data to a control unit 170 described later.

In the present embodiment, the drowsiness determination device is mounted on the vehicle, and the camera used in the drowsiness determination device is also used as the face image acquisition means 160 in the travel control device 100.

The drowsiness determination device acquires a driver's face image with a camera in time series, and from the obtained face image, the driver's state, for example, the driver's eye openness, the direction of gaze, It is an apparatus that assists in understanding driving direction, concentration on driving, etc., and leading to safe driving. Specifically, from the driver's face image obtained by the camera, for example, it may be determined whether or not the driver is drowsy by grasping the degree of eye opening, and the driver seems to be drowsy. For example, appropriate driving assistance such as activating an alarm or the like to awaken the driver or decelerating or forcibly stopping the brake device is performed.

The control unit 170 is configured mainly by a computer, and includes the positioning information detection unit 110, the travel route information detection unit 120, the surrounding vehicle information detection unit 130, the surrounding environment recognition unit 140, and the own vehicle information collection unit 150. Accelerator opening and steering angle for automatic driving control of the vehicle based on various information (positioning information, travel route information, surrounding vehicle information, surrounding environment information, own vehicle information), and preset programs In addition, it has a function of determining each control value for the brake depression force and the like.

Based on the determined control values, the controller 170 controls the accelerator opening control device 210 that controls the accelerator opening, the steering angle control device 220 that controls the steering angle by steering, and the brake that controls the effectiveness of the brake. By performing drive control of the control device 230, automatic driving control of the vehicle is realized. The above accelerator, steering, brake, and the like correspond to the drive unit.

In addition, the control unit 170 performs a caution action when the driver tries to change the traveling state of the vehicle in accordance with the change in the face-series face orientation in the face image obtained from the face image acquisition unit 160. It is determined whether or not the vehicle is in safe driving mode (details will be described later). The control unit 170 is also a determination unit that determines the presence or absence of the driver's attention behavior.

The operation of the travel control device 100 formed as described above will be described below with reference to FIG.

First, the control unit 170 performs automatic operation control. That is, the control unit 170 always collects positioning information from the positioning information detecting unit 110 in real time, collects driving route information from the driving route information detecting unit 120, collects surrounding vehicle information from the surrounding vehicle information detecting unit 130, The surrounding environment information is collected from the surrounding environment recognition unit 140, and the own vehicle information is collected from the own vehicle information collection unit 150.

Next, the control unit 170 determines control values for the accelerator, the steering, and the brake based on the collected various information (positioning information, travel route information, surrounding vehicle information, surrounding environment information, and own vehicle information). And the control part 170 performs drive control with respect to the accelerator opening degree control apparatus 210, the steering angle control apparatus 220, and the brake control apparatus 230 based on each determined control value, The automatic driving | operation of the said vehicle Take control.

Furthermore, the control unit 170 performs measure control for safe driving based on the flowchart shown in FIG.

First, in S100, the control unit 170 obtains a face image from the face image obtaining unit 160 at predetermined intervals, and grasps a change in the orientation of the face, thereby determining whether or not there is a driver's attention behavior during driving. Monitor. For example, when the nose (mouth) of the acquired face image is positioned substantially at the center in the left-right direction, the driver is regarded as facing the front. Further, for example, when the nose (mouth) of the acquired face image is located on the left side of the image, the driver is assumed to be facing the right side. Furthermore, for example, when the nose (mouth) of the acquired face image is located on the right side of the image, the driver is assumed to be facing the left side. In this way, the control unit 170 grasps the change in the face direction by setting three zones, which are the center, right, and left of the driver's face area. The zones are not limited to three, and can be set arbitrarily. In addition, as described above, the orientation of the face is not limited to the nose (mouth) of the face image, but may be determined using the positional relationship on the face image such as the eyes, eyebrows, and cheeks. .

When the driver is facing the front, it is understood that he is driving normally while looking at the front of the vehicle. Also, when the driver is looking to the right or left, for example, it is understood that the right or left lane status is confirmed (attention action is being performed) in order to change the driving lane (lane change). Is done.

In S110, the control unit 170 determines whether or not the driver is performing a caution action based on the change in face orientation as described above. As a procedure for determination, when the driver's face direction changes from the front to the right or left for a predetermined time or more, the driver is looking to the right or left. Considering that the lane situation is confirmed, it is determined that there is a caution action. Conversely, if the time when the driver's face changes is less than the predetermined time, the driver has not intentionally turned to the right or left, and has not confirmed the right or left lane status. It is determined that there is no caution action. S110 is a determination step in which the control unit 170 (determination unit) determines whether or not there is a caution action.

If it is determined that there is a caution action in S110 (S110: Yes), the control unit 170 permits the operation of the drive unit related to the change of the traveling state in the automatic driving control in S120. For example, the steering angle control device 220 (steering as a drive unit) is allowed to change the steering angle for the lane change.

On the other hand, when it is determined in S110 that there is no caution action (S110: No), the control unit 170 prohibits the operation of the drive unit related to the change of the traveling state in the automatic driving control in S130. For example, the steering angle change for the lane change is prohibited for the steering angle control device 220 (steering as a drive unit).

As described above, in the present embodiment, the control unit 170 (determination unit) determines whether or not there is a caution action when the driver tries to change the driving state of the vehicle during the automatic driving control in a time-series manner in the face image. Judgment is based on changes in the orientation of the face. For example, in the configuration in which the degree of attention paid by the driver as in the prior art is detected based on the driver's gaze direction, the movement of the human gaze can be detected by grasping the movement of the eye pupil. The moving range of the pupil is a small range limited in the open eye. On the other hand, in the present embodiment in which the presence or absence of the driver's attention behavior when changing the running state is determined from the time-series change in the face direction in the face image, a small change in the gaze direction Therefore, the change in the orientation of the face can be greatly obtained as compared with the case of determining from the above, and the determination accuracy can be remarkably improved. Then, the control unit 170 determines whether the driving unit (for example, steering) according to the change of the running state (for example, lane change) in the automatic driving control when the attention behavior for a predetermined time or more is not recognized from the change of the face direction. Since the operation prohibition is executed, the safe driving operation can be surely realized.

In the present embodiment, the travel control device 100 is applied to a vehicle equipped with a drowsiness determination device, and the face image acquisition unit 160 is also used as a camera of the drowsiness determination device. . As a result, an increase in the number of newly installed parts can be reduced, and inexpensive measures can be taken.

In addition, the control unit 170 determines that the attention action is performed when the change in the face orientation is equal to or longer than the predetermined time, and that the attention action is not performed when the change is less than the predetermined time. Thus, it is possible to clearly determine the presence or absence of attention behavior.

In addition, since the control unit 170 regards the change in the left and right face orientation as a confirmation action to the left and right lanes, this allows the driver to understand the attention behavior at the time of the lane change. It is possible to ensure safety during the change.

(Second Embodiment)
A travel control apparatus 100A of the second embodiment is shown in FIGS. The travel control device 100A of the second embodiment is obtained by adding a warning unit 180 to the travel control device 100 of the first embodiment. The flowchart shown in FIG. 4 is obtained by adding S125 to the flowchart described in FIG.

The warning unit 180 generates a warning sound such as an alarm for the driver, or by displaying characters such as “Please check left and right” on the information display unit or the navigation screen in the combination meter unit, This is a warning means that warns the user to perform the caution action while driving.

In the flowchart shown in FIG. 4, when the control unit 170 determines in S110 that the driver is not taking the caution action in executing the measure control for the safe driving, first, before the execution of S130, the control unit S125. Then, the warning unit 180 is activated.

As a result, a warning due to insufficient attention behavior is transmitted to the driver before the operation of the drive unit for changing the driving state is prohibited, so the driver can operate the driving unit for changing the driving state. It is possible to respond to the prohibition of the problem without any sense of incongruity.

(Third embodiment)
The control procedure of the third embodiment is shown in FIG. The flowchart shown in FIG. 5 is obtained by eliminating S120 and S130 from the flowchart described in FIG.

When the driver's attention behavior is not recognized, the first embodiment prohibits the operation of the drive unit for changing the driving state, and the second embodiment warns the driver and changes the driving state. The operation of the drive unit is prohibited. In the present embodiment, only a warning is given to the driver in S125 after a negative determination in S110 (when the driver's attention behavior is not recognized). Thus, even if it is only a warning by the driver, it can be a measure for safe driving.

(Fourth embodiment)
A control procedure of the fourth embodiment is shown in FIG. The flowchart shown in FIG. 6 is an example showing how measure control for safe driving is performed with respect to specific control contents incorporated in advance in automatic driving control.

First, in S10, the control unit 170 determines whether or not the vehicle is under automatic driving control. If not in automatic operation, the subroutine is terminated, and if automatic operation is in progress, the process proceeds to S20.

In S20, in the automatic driving control, the control unit 170 determines whether or not parking is scheduled due to a lane change, a start from a stop state by a signal, or a reverse within a predetermined time. If these running state change controls are not scheduled, the subroutine is terminated, and if scheduled, the process of S30 is performed.

In S30, the controller 170 prompts the driver to perform a caution action commensurate with the driving state change control, and proceeds to S40. Specifically, when the travel state change control is a lane change, the driver is notified that the vehicle is about to change the lane and the direction of the lane change. Further, if the traveling state change control is a start from a stop state by an intersection signal, the driver is notified that the own vehicle will start. Further, if the traveling state change control is a parking by reverse, the rear view monitor is activated to urge the driver to pay attention to the rear of the vehicle.

In S40 and S110, the face image of the driver who is driving the vehicle is acquired by the face image acquisition means 160, and the presence / absence of the driver's attention behavior is determined according to the time-series change in face orientation in the face image. Determination is performed by the determination unit (S110). The determination unit recognizes the direction and line of sight of the driver's face, and determines whether the driver is paying attention to the direction according to the driving state change control. Specifically, if it is a lane change, whether it is paying attention to the side of the vehicle in the direction of the lane change, if the vehicle starts from a stop state due to a signal, or if the vehicle is parked by reversing, the vehicle rearward Determine.

This determination of presence / absence of attention behavior is made when the behavior of the face has changed for a certain period of time or when the number of times the face orientation has changed is greater than or equal to a predetermined number of times. To do.

If it is determined by S110 that the driver has taken the caution action, the process proceeds to S120, and if it is determined that the caution action has not been taken, the process proceeds to S126.

In S120, the control unit 170 determines whether to permit planned lane change, starting from a stop state by a signal, or parking by reversing (accelerator opening control device 210, steering angle control). A signal to be transmitted to the device 220 and the brake control device 230) is transmitted, and the subroutine is terminated.

In S126, it is determined whether or not a predetermined time has elapsed from the notification in S30. If the predetermined time has not elapsed, the process returns to S40, and if the predetermined time has elapsed, the process proceeds to S130A. Before returning to S40, the warning unit 180 may be activated to warn the driver not to neglect the surroundings even during the automatic driving control.

In S130A, the operation of the drive unit for changing the traveling state is prohibited or the operation content is changed, and the subroutine is terminated. Specifically, it is possible to cancel lane changes, or do not complete lane changes at once, and perform them in multiple stages gently, or start and reverse parking from a signal at a slower speed than usual. .

Thereby, the same effect as the first embodiment or the second embodiment can be obtained.

In the present embodiment, a drowsiness determination device that recognizes whether or not the driver is drowsy from the driver's face image captured by the face image acquisition unit 160 may be further provided.

(Other embodiments)
In each of the above embodiments, the drowsiness determination device is mounted on the vehicle, and the facial image acquisition unit 160 of the

travel control devices

100 and 100A is also used as the camera of the drowsiness determination device. However, the present invention is not limited to this. Alternatively, the face image acquisition means 160 dedicated to the

travel control devices

100 and 100A may be provided without the drowsiness determination device.

In addition, the control unit 170 determines whether or not there is a change in the orientation of the face with a predetermined time. However, the control unit 170 is not limited thereto, and may be determined with a predetermined number of times. In other words, when the face orientation continuously changes more than a predetermined number of times, it is determined that there is a driver's attention behavior (checking left and right), and if the number of face direction changes is less than the predetermined number of times Further, it may be determined that there is no driver's attention behavior (the left and right are not confirmed).

In the above-described embodiment, for example, S100 that acquires the driver's face image data from the face image acquisition unit 160 corresponds to the face image acquisition unit, and responds to changes in face orientation in time series based on the acquired face image data. S110 for determining the presence or absence of the driver's attention behavior corresponds to the attention behavior determination unit, and when the attention behavior determination unit determines that there is no driver's attention behavior, S125 or driving state change control for operating the warning unit S130 and S130A which perform are equivalent to an operation control part. As the running state change control, for example, at least one of the accelerator opening degree control device 210, the steering angle control device 220, and the brake control device 230 is changed or prohibited in the control contents related to the running state change.

Here, the flowchart described in this application or the process of the flowchart is configured by a plurality of sections (or referred to as steps), and each section is expressed as S100, for example. Furthermore, each section can be divided into a plurality of subsections, while a plurality of sections can be combined into one section. Further, each section configured in this manner can be referred to as a device, module, or means.

Claims

A controller (170) that performs automatic driving control of the vehicle based on various information required for traveling;
Face image acquisition means (160) for acquiring a face image of a driver driving the vehicle;
A determination unit (170, S110) that determines the presence or absence of the driver's attention behavior according to a change in face orientation in time series in the face image,
The control unit (170) operates the warning unit (180) for warning the driver and changes the driving state in the automatic driving control when the attention behavior is not recognized for a predetermined time. A travel control device that executes at least one of operation prohibition or operation content change.
In the automatic driving control, the determination unit determines whether or not there is a driver's attention action when parking from a lane change, a start from a stop state by a signal, or a reverse is scheduled, and the control unit When at least one of the caution actions is not recognized for a predetermined time, at least one of the operation of the warning unit (180) for warning the driver and the change of the operation content of the drive unit according to the change of the driving state in the automatic driving control The travel control device according to claim 1, wherein:
The determination unit determines that the attention action is performed when the state in which the orientation of the face is changed continues for a certain time or when the number of changes in the orientation of the face is a predetermined number or more. The travel control device according to claim 1 or claim 2.
The drowsiness determination device according to any one of claims 1 to 3, further comprising a drowsiness determination device that grasps whether or not the driver is drowsy from the face image of the driver taken by the face image acquisition unit (160). The travel control device according to one.
The change of the running state includes at least one of acceleration, deceleration, change of driving direction, lane change of the vehicle,
The travel control device according to any one of claims 1 to 4, wherein the control unit performs operation control on at least one of an accelerator opening control device, a steering angle control device, and a brake control device.
A travel control device that performs automatic driving control of a vehicle,
A face image acquisition unit for acquiring driver's face image data;
Based on the acquired face image data, an attention behavior determination unit that determines the presence or absence of the driver's attention behavior according to a change state of the driver's time-series facial orientation;
When the attention behavior determination unit determines that there is no driver's attention behavior, at least one of an accelerator opening control device, a steering angle control device, and a brake control device that controls operation of the warning unit and automatic driving An operation control unit that performs at least one of the operation control including prohibition or change of the operation related to the change of the traveling state,
A travel control device comprising:
A driving state change determination unit that determines whether or not parking is planned due to a lane change, a start from a stop state by a signal within a predetermined time, or a reverse;
The attention behavior determination unit determines whether the planned driving state is determined when the driving state change determination unit determines that a lane change, start from a stop state by a signal, or parking by a reverse is scheduled within a predetermined time. The travel control device according to claim 6, wherein the presence or absence of the driver's attention behavior according to a change is determined.