JP2006160032A - Driving state determination device and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driving state determination device and its method capable of appropriately determining the driving state of a vehicle. <P>SOLUTION: The driving state determination device which is mounted to the vehicle and determines the driving state of the vehicle, has a vehicle state detection means detecting various information showing the state of each part of the vehicle, an external environment detection means detecting various information related to the external environment of the vehicle, and a driving state determination means which determines the driving state of the vehicle using a neural network calculating the occurrence probability of each driving state by making various information showing the state of each part of the vehicle and various information related to the external environment as the input information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a driving state determination device and a driving state determination method, and more particularly to a driving state determination device and a driving state determination method for determining a driving state of a vehicle such as an automobile.

A technology that automatically determines the driving status of vehicles such as lane changes, left and right turns, and parking operations in order to realize driving recorders that are currently being considered to be legally required to be mounted mainly in the United States. Development is strongly required. In providing services such as a driving support system and an agent, it is possible to provide a more advanced and comfortable application by determining the driving state in a timely and accurate manner.
JP 2000-233699 A JP 2004-53267 A

  However, most of the technologies already disclosed are not intended to determine the driving state itself, but only to detect a specific or partial driving state for the purpose of providing a certain service or the like.

  For example, the technique described in Patent Document 1 detects a part of the driving state such as a right or left turn for the purpose of detecting an object in the traveling direction of the vehicle.

  On the other hand, in Patent Document 2, “deceleration”, “stop”, “turn”, “change of course”, “avoidance”, “reverse” are obtained from operation data of brake, accelerator, steering angle, shift operation, and winker. , “Turn left / right”, and “join” are described. However, the disclosed content of Patent Document 2 has a problem that it is difficult to accurately distinguish and determine, for example, “turn”, “lane change”, “right / left turn”, and “merge”. That is, with respect to “lane change” and “right / left turn”, both can occur even when the winker operation, the steering angle, the brake operation, the accelerator operation, and the shift operation are all the same. Further, for example, even when a “right / left turn” is performed without a winker operation, it is difficult to distinguish between “right / left turn” and “turn”.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a driving state determination device and a driving state determination method that can appropriately determine the driving state of a vehicle.

  Accordingly, in order to solve the above-described problem, the present invention is an operation state determination device that is mounted on a vehicle and determines the operation state of the vehicle, as described in claim 1, wherein the state of each part of the vehicle is determined. Vehicle state detection means for detecting various information shown, external environment detection means for detecting various information related to the external environment of the vehicle, various information indicating the state of each part of the vehicle, and various information related to the external environment And a driving state determination means for determining the driving state of the vehicle using a neural network that calculates the occurrence probability of each driving state using the above as input information.

  In such a driving state determination device, since the driving state is determined using a neural network that uses not only the state of each part of the vehicle but also information related to the external environment as input information, the driving state of the vehicle is appropriately determined. Can do.

  Moreover, in order to solve the said subject, this invention is good also as the driving | running state determination method in the said driving | running state determination apparatus.

  ADVANTAGE OF THE INVENTION According to this invention, the driving | running state determination apparatus and driving | running state determination method which can determine the driving | running state of a vehicle appropriately can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration example of an operation state determination system according to an embodiment of the present invention. In FIG. 1, a driving support system 1 is a system that is mounted on a vehicle such as an automobile and automatically determines the driving state of the vehicle to provide a service corresponding to the driving state to the driver. The ECU 11, the vehicle information recognition ECU 12, the determination processing ECU 13, the driving state recording ECU 14, the driving support ECU 15, and the like, the various sensors connected to the ECUs, the driving record DB (database) 16, and the like. The Here, the ECU (Electronic Control Unit) is constituted by a microcomputer, for example, a ROM for storing a control program, a readable / writable RAM for storing calculation results, a timer, a counter, an input interface, and an output interface. And the like.

  The environmental information recognition ECU 11 includes an inter-vehicle range sensor 111, a lane image recognition sensor 112, a signal image recognition sensor 113, a sign image recognition sensor 114, a temporary stop line image recognition sensor 115, a pedestrian image recognition sensor 116, a vehicle periphery sensor 117, Based on input information from the traffic environment information acquisition communication device 118, the car navigation system 119, and the like, various types of information related to the external environment of the vehicle, particularly information that restricts the traveling of the vehicle in some form (hereinafter referred to as "environment information"). ECU) that recognizes.

  The inter-vehicle range sensor 111 is a sensor that detects the inter-vehicle distance from the vehicle ahead. The lane image recognition sensor 112 is a sensor that detects the distance from the lane based on the lane image input from the camera. The signal image recognition sensor 113 is a sensor that detects the color of the signal based on the image of the front signal input from the camera. The sign image recognition sensor 114 is a sensor that identifies the meaning of the traffic sign based on the image of the front traffic sign input from the camera. The pause line image recognition sensor 115 is a sensor that detects the distance to the pause line based on the pause line image input from the camera. The pedestrian image recognition sensor 116 is a sensor that detects the presence of a pedestrian based on a forward pedestrian image input from a camera. The vehicle surrounding sensor 117 is a sensor that detects an obstacle or the like around the vehicle. The traffic environment information acquisition communication device 118 is a device that acquires traffic environment information such as an accident or traffic jam by communication. The car navigation system 119 is a so-called car navigation system, and is a system for determining the current position of the vehicle and the road form (straight line, intersection, T-junction, etc.) currently drafted from map data or the like.

  The vehicle information recognition ECU 12 is a variety of information indicating the state of each part of the vehicle based on input information from the vehicle speed sensor 121, the steering angle sensor 122, the accelerator opening sensor 123, the brake depression degree sensor 124, the winker operation sensor 125, and the like. It is an ECU that recognizes (hereinafter referred to as “vehicle information”).

  The vehicle speed sensor 121 is a sensor that detects the vehicle speed. The steering angle sensor 122 is a sensor that detects the steering angle. The accelerator opening sensor 123 is a sensor that detects the degree of depression of the accelerator. The brake depression degree sensor 124 is a sensor that detects the degree of depression of the brake. The winker operation sensor 125 is a sensor that detects the operation of the winker.

  The determination processing ECU 13 is an ECU that determines the driving state of the vehicle based on the environmental information recognized by the environmental information recognition ECU 11 and the vehicle information recognized by the vehicle information recognition ECU 12.

  The driving state recording ECU 14 records the driving state determined by the determination processing ECU 13 in the driving record DB 16 and determines when the determination result by the determination processing ECU 13 is incorrect (when different from the actual driving state). The ECU corrects the determination logic in the process ECU 13.

  The driving support ECU 15 is an ECU that provides a driving support service corresponding to the driving state determined by the determination processing ECU 13 via, for example, the display 151, the speaker 152, the actuator 153, or the like.

  In the present embodiment, the determination processing of the driving state by the determination processing ECU 13 is logicalized based on the definition of the driving state hierarchically subdivided. FIG. 2 is a diagram illustrating a definition example of an operation state in the embodiment of the present invention.

  In the table 201 shown in FIG. 2, the operation state is subdivided into three layers from L1 (level 1) to L3. As the largest category L1, “forward”, “reverse”, and “stop” are defined. In L2, which is a subdivision of L1, “straight forward” and “turn” for “forward”, “straight reverse” and “back turn” for “reverse”, “pause”, “stop” for “stop” (Idling state) "and" parking "are defined respectively. Furthermore, L3 is divided into L2, and “Acceleration”, “Constant speed” and “Deceleration” are defined for “Straight”, “Turn (curve)”, “Turn left and right” and “Change lane” for “Turn” Has been.

  In Table 201, input information having a high weight is defined in determining each of the operating states of L2 or L3. For example, in order to determine acceleration (L3), it is defined that information on vehicle speed, accelerator, brake, and steering angle is important. The operation state determination logic incorporated in the determination processing ECU 13 based on this definition is, for example, as shown in FIG. FIG. 3 is a flowchart for explaining the determination process of the driving state by the determination process ECU.

  In step S101 of FIG. 3, it is determined whether or not IG (ignition) is OFF based on the vehicle information from the vehicle information recognition ECU 12. When it is determined that the ignition is OFF (Yes in S101), it is determined that the driving state is the “parking” state (S102). If it is determined that the ignition is not OFF (No in S101), the process proceeds to step S103, and it is determined whether the vehicle speed is 0 (km / h) based on the vehicle information. When it is determined that the vehicle speed is 0 (Yes in S103), the process proceeds to step S104, and it is determined whether or not the vehicle has stopped for a predetermined time (here, α minutes) based on the vehicle information. If it is determined that the vehicle has stopped for more than α minutes (Yes in S104), it is determined that the driving state is a “stop” state (S105). Otherwise (No in S104), the driving state is “temporarily stopped”. ] State (S106).

  On the other hand, when it is determined in step S103 that the vehicle speed is not 0 (No in S103), the process proceeds to step S107, and it is determined whether or not the shift position is R (Rear) based on the vehicle information. When it is determined that the shift position is R (Yes in S107), it is determined that the driving state is the “reverse” state (S108). If it is determined that the shift position is not R, the process proceeds to step S109 to determine whether or not the steering angle is near 0 (degrees) based on the vehicle information.

If it is determined that the steering angle is near 0 (Yes in S109), it is determined that the driving state is the “straight-ahead” state (S110), and further, a step is performed to specify the lower-level driving state belonging to “straight-ahead”. The processing from S111 to S115 is performed. That is, when it is determined that the acceleration is near 0 (km / s ² ) (Yes in S1111), it is determined that the state is “constant speed” (S112). If it is determined that the acceleration is not near 0 (No in S111), it is determined whether the acceleration is positive (+) (S113). If the acceleration is positive, the driving state is an “acceleration” state. It judges (S114), and when that is not right, it determines with a driving | running state being a "deceleration" state (S115).

  On the other hand, if it is determined in step S109 that the steering angle is not near 0 (No in S109), the process proceeds to step S116, and it is determined whether the winker is operating based on the vehicle information. When it is determined that the winker is operating (Yes in S116), the process proceeds to step S117, and it is determined whether the steering angle is large based on the vehicle information. As will be described later, it is determined whether the vehicle is “right / left turn” or “lane change” depending on the size of the steering angle. Therefore, the criterion for determining whether or not the steering angle is large may be determined based on experience values or the like so that the right / left turn and the lane change can be distinguished.

  If it is determined that the steering angle is large based on this criterion (Yes in S117), the process proceeds to step S118, where it is determined that the driving state is “turn left / right”, and “turn right” based on the winker state. Or “left turn”. That is, if the turn signal indicates the right, it is determined as “right turn” (S120), and otherwise, it is determined as “left turn” (S121).

  When it is determined in step S117 that the steering angle is not large (No in S117), the process proceeds to step S112, where it is determined that the driving state is “lane change”, and further, the right lane is changed based on the blinker state. Or whether it is a change to the left lane. That is, when the turn signal indicates the right, it is determined that the lane change is “to the right lane” (S124). Otherwise, it is determined that the lane change is “to the left lane” (S125).

  On the other hand, if it is determined in step S116 that the winker is not operating (No in S116), it is determined that the driving state is one of “lane change”, “right / left turn”, or “turn” (S127), Further, the left / right distinction is determined based on the steering angle (S127). That is, when the steering angle is rightward (Yes in S127), it is determined that the lane is changed to “turn right”, “turn right” or “to right lane” (S128), and otherwise (S127). No), it is determined that the lane change is “turn left”, “turn left” or “to left lane” (S129).

  As described above, in FIG. 3, various driving states are determined mainly by conditional branching (IF-THEN rule) based on vehicle information.

  By the way, the ambiguity remains in the determination result of the driving state by the process of FIG. 3 in steps S128 and S129. That is, the distinction between “turn”, “turn left / right” or “change lane” is not specified. One of the reasons for this distinction is that it requires delicate judgments and is easily affected by driver's habits. Therefore, the determination processing ECU 13 in the present embodiment determines these operating states using a neural network.

  FIG. 4 is a diagram illustrating an example of a neural network for determining a lane change state. The neural network 131 of FIG. 4 is defined as input information with high weight for “lane change” in FIG. 2, steering angle, vehicle speed, winker, shift position, map data of the car navigation system 119, and lane image recognition sensor. The lane recognition result by 113 is used as input information or as highly weighted input information. In general, in each neuron in the neural network, firing (output) is performed when the sum of input values exceeds a threshold value, but in the neuron of the input layer of the neural network 131 (the neuron in the leftmost column in the figure). For example, the waveform information of the graph representing the transition within the past fixed time (for example, 30 seconds to 1 minute) immediately before the determination is used as the input information. And when the similarity (for example,%) of the waveform information and the waveform information predetermined as a threshold value is high, ignition is performed. For example, “vehicle speed” as input information corresponds to waveform information of a graph representing a transition in vehicle speed until immediately before the determination. Then, the vehicle speed waveform information is compared with the vehicle speed waveform information set as a threshold value in advance. When the similarity is equal to or higher than a predetermined value, 1 is output from the neuron, and 0 is output otherwise. That is why.

  The similarity of the waveform information may be determined using various known methods such as residual matching, normalized correlation method, phase-only correlation, geometric matching, vector correlation, or generalized Hough transform. However, regarding time-series charts such as vehicle speed, since there is a large variation in the vehicle speed level, geometric matching that is not easily affected by resizing is desirable. For time series charts such as shift changes, simple residual matching is desirable.

  Each connection between the input layer and the intermediate layer and each connection between the intermediate layer and the output layer are weighted in advance, and in the intermediate layer, the sum of the weighted values for the input from the input layer. The output is performed based on the same as in a general neural network.

  The determination processing ECU 13 uses the neural network 131 to calculate the occurrence probabilities of “right turn”, “right turn”, and “to the right lane”, and determine the driving state with the highest occurrence probability. Adopt as a result.

  Incidentally, the operating states defined as L1 to L3 in FIG. 2 can be further subdivided. However, as the subdivision progresses, the difference in each driving state becomes more subtle and more susceptible to the influence of the driver's habit. Therefore, the determination processing ECU 13 in this embodiment determines not only the cases of steps S128 and S129 but also the subdivided operation state using a neural network. Hereinafter, typical examples will be described.

  FIG. 5 is a diagram illustrating a definition example of the operation state in the lower layer of the temporary stop. In the table 202 of FIG. 5, the “pause” defined as L2 in the table 201 (FIG. 2) is further classified based on the cause, and each of the classified operation states is defined as L4. Yes. That is, whether it is “by a signal”, “by a stop sign”, “by a preceding vehicle (other vehicle)”, “by a person / obstacle”, or “ It is classified as “other”. The determination of L4 related to the temporary stop is performed using, for example, a neural network as shown in FIG.

  FIG. 6 is a diagram illustrating an example of a neural network for determining a paused state. The neural network 132 in FIG. 6 is defined as input information having a high weight in the table 202. The vehicle speed, the shift position, the time when the vehicle speed = 0, the signal recognition result by the signal image recognition sensor 113, and the sign image recognition sensor 114. As a result of the sign recognition, the result of recognition of other vehicles by the inter-vehicle range sensor 111, the result of recognition of pedestrians and obstacles by the pedestrian image recognition sensor 116 and the vehicle periphery sensor 117, etc. as input information or high weighting Used as input information. The determination processing ECU 13 uses the neural network 132 to select “by signal”, “by stop sign”, “by front vehicle (other vehicle)”, “by person / obstacle”, and “by other”. The occurrence probability is calculated, and the driving state with the highest occurrence probability is adopted as the determination result.

  Moreover, FIG. 7 is a figure which shows the example of a definition of the driving state of the lower layer of parking operation. In Table 203 of FIG. 7, “straight forward”, “turn”, “straight reverse”, “back turn”, “pause”, “stop” (idling state) defined as L2 in Table 201 (FIG. 2) The lower layer of “Straight” and “Turn” are the lower layers of “Acceleration”, “Constant velocity”, “Deceleration”, “Turning (curve)”, “Turn left / right”, “Change lane” ), A driving state “parking operation” is commonly defined in L4, and “back-in”, “column”, and “front-in” are defined as lower layers (L5) of the “parking operation”. The determination of L5 related to “parking operation” is performed using, for example, a neural network as shown in FIG.

  FIG. 8 is a diagram illustrating an example of a neural network for determining the state of the parking operation. The neural network 133 in FIG. 8 is defined as input information having a high weight in the table 203, the transition state of the driving state of L2 (L3 for “straight” and “turn”), the shift position, and the vehicle surrounding sensor 117. Vehicle periphery information from the vehicle, navigation information from the car navigation system 119 (for example, parking lot position information), and the like are used as input information or highly weighted input information. The determination processing ECU 13 uses the neural network 133 to calculate the occurrence probabilities of “back insertion”, “column”, and “front insertion”, and adopts the driving state with the highest occurrence probability as the determination result.

  The state of transition of the driving state such as L2 is, for example, information (for example, waveform information) indicating the timing at which each driving state occurs in a predetermined time (for example, 30 seconds to 1 minute) immediately before the determination. Corresponds.

  Furthermore, FIG. 9 is a figure which shows the example of a definition of the driving | running state of the lower layer of a lane change. In the table 204 of FIG. 9, as the lower layer (L4) of “lane change” defined as L3 in the table 201 (FIG. 2), “to overtaking lane”, “join”, “to driving lane”, and “ "Parallel parking" is defined. The determination of L4 related to “lane change” is performed, for example, using a neural network as shown in FIG.

  FIG. 10 is a diagram illustrating an example of a neural network for determining the state of lane change. The neural network 134 in FIG. 10 includes the steering angle, vehicle speed, blinker state, shift position, navigation information, lane recognition result from the lane image recognition sensor 112, and the like, which are defined as high-weight input information in Table 204. Used as input information or highly weighted input information. The determination processing ECU 13 uses the neural network 134 to calculate the occurrence probabilities of “to overtaking lane”, “join”, “to driving lane”, and “parallel parking”, and to determine the driving state with the highest occurrence probability. Adopt as a judgment result.

  The driving support ECU 15 can provide various services according to the driving state determined by the determination processing ECU 13. For example, when it is determined as “parking operation”, a vehicle peripheral image may be displayed on the display 151. Further, when it is determined as “pause” that does not affect driving, a massage sheet may be activated or an incoming mail may be announced. Furthermore, when it is determined that “lane change”, an alarm may be sounded according to the distance from the rear vehicle at the lane change destination.

  The determination results made using various neural networks are also output to the driving state recording ECU 14 and recorded in the driving record DB 16 by the driving state recording ECU 14. The driving state recording ECU 14 also detects a difference between the determination result and the actual driving state, and corrects the weighting of the connection between each layer (input layer, intermediate layer and output layer) of the neural network based on the difference. Do (Neural network learning). The actual driving state can be specified by, for example, a travel locus in the navigation information. That is, when it is determined that the right turn is made by the determination processing ECU 13, but it is confirmed that the lane has been changed in the travel locus in the navigation information, the related neural network is learned using the navigation information as a teacher signal.

  The learning logic may be performed using a well-known method that has already been established (for example, the back-propagation method (BP method)).

  As described above, according to the driving state determination system 1 in the present embodiment, not only information on the vehicle itself (vehicle information) such as vehicle speed, steering angle, or accelerator opening, but also information on the external environment outside the vehicle (environment) (Information) is also used as input information, and because the driving state is determined by the determination logic based on the hierarchically divided driving state definition, the determination result specified to a fine level is automatically output. can do. In addition, since the operation state is determined using the neural network, it is possible to obtain a highly appropriate determination result for the operation state that requires delicate determination. In addition, the learning function can feed back an error in the determination result to the determination logic (weighting of the neural network), so that a determination corresponding to a driver's habit or the like is possible.

  In the present embodiment, an example in which up to L3 is determined by a flowchart, that is, by a conditional branch (IF-THEN rule), and the following is determined by a neural network has been described. The determination may be made by conditional branching to a lower level. However, in this case, it is difficult to obtain a determination result reflecting the driver's habit.

  On the other hand, the determination may be made by the neural network from the upper level (L1) without using the conditional branch. However, computation using a neural network generally requires a high load and tends to slow down the processing speed. Therefore, as in the present embodiment, a method of determining by a conditional branch up to a certain level, and using a neural network for a lower level, is a viewpoint of harmony between high speed of processing and flexibility of determination. Desirable from.

  However, it is not always necessary to set the boundary between the conditional branch and the neural network between L3 and L4 as in the present embodiment. Which level is used as the boundary between the conditional branch and the neural network may be determined according to characteristics required for the system. In other words, the wider the range determined by conditional branching, the faster and uniform the determination process, while the wider the range determined by the neural network, the more flexible the driver's habits The possibility that a determination result having sex and diversity is obtained increases.

  Note that the present invention is not limited to such specific embodiments, and various modifications and changes can be made within the scope of the gist of the present invention described in the claims.

It is a figure which shows the structural example of the driving | running state determination system in embodiment of this invention. It is a figure which shows the example of a definition of the driving | running state in embodiment of this invention. It is a flowchart for demonstrating the determination process of the driving | running state by determination process ECU. It is a figure which shows the example of the neural network for determining the state of lane change. It is a figure which shows the example of a definition of the driving | running state of the lower layer of a temporary stop. It is a figure which shows the example of the neural network for determining the state of a temporary stop. It is a figure which shows the example of a definition of the driving | running state of the lower layer of parking operation. It is a figure which shows the example of the neural network for determining the state of parking operation. It is a figure which shows the example of a definition of the driving | running state of the lower layer of a lane change. It is a figure which shows the example of the neural network for determining the state of lane change.

Explanation of symbols

1 Operating state determination system 11 Environmental information recognition ECU
12 Vehicle information recognition ECU
13 judgment processing ECU
14 Operating state recording ECU
15 Driving assistance ECU
16 Operation record DB
111 Vehicle-to-vehicle range sensor 112 Lane image recognition sensor 113 Signal image recognition sensor 114 Sign image recognition sensor 115 Pause line image recognition sensor 116 Pedestrian image recognition sensor 117 Vehicle periphery sensor 118 Communication device 119 for obtaining traffic environment information Car navigation system 121 Vehicle speed sensor 122 Rudder angle sensor 123 Accelerator opening sensor 124 Brake depression sensor 125 Blinker operation sensor 151 Display 152 Speaker 153 Actuator

Claims (5)

A driving state determination device that is mounted on a vehicle and determines a driving state of the vehicle,
Vehicle state detecting means for detecting various information indicating the state of each part of the vehicle;
An external environment detection means for detecting various information related to the external environment of the vehicle;
Driving state determination for determining the driving state of the vehicle using a neural network that calculates the occurrence probability of each driving state using various information indicating the state of each part of the vehicle and various information regarding the external environment as input information And a driving state determination device. The driving state determination device according to claim 1, further comprising learning means for learning the neural network using an actual driving state as a teacher signal. The driving state determination device according to claim 1, wherein the actual driving state is determined based on information indicating a traveling locus of the vehicle by a car navigation system mounted on the vehicle. Each operating state is defined in advance hierarchically subdivided,
The said driving | running state determination means determines the driving | running state to a predetermined | prescribed hierarchy by the conditional branch based on the various information which shows the state of each part of the said vehicle at least. Driving state determination device. A driving state determination method for determining a driving state of a vehicle,
A vehicle state detection procedure for detecting various information indicating the state of each part of the vehicle;
An external environment detection procedure for detecting various information related to the external environment of the vehicle;
Driving state for determining the driving state of the vehicle using a neural network that calculates the occurrence probability of each driving state by using various information indicating the state of each part of the vehicle and various information regarding the external environment as input information An operating state determination method comprising: a determination procedure.

Images