TW201511997A - Autonomous automatic driving auxiliary system and method - Google Patents

Abstract

The present invention relates to an autonomous automatic driving auxiliary system and method. This system is integrated with the control system of vehicle, continues detecting the surrounding environment of a vehicle to find out the cars travelling in the same direction as the present car, and proceeds in an automatic driving way to follow the trail of the cars in order to automatically drive to the destination. This system is operated mainly through light signal identification of the front car's direction lights and early determination of the front car's driving direction and driving state to reduce emergency brake and vehicle collision and improve driving efficiency. This invention does not need to install expensive radar detection equipment and is readily integrated with vehicle control system, thereby solving the problems of expensive setup cost of existing autonomous automatic driving auxiliary device and the difficulty of system integration.

Description

Active automatic driving assistance system and method

The invention relates to an automatic driving assistance system and method, in particular to an automatic driving assistance system and method integrating road environment detection, automatic driving, vehicle transportation and vehicle body safety integration technology.

Existing vehicles have been equipped with more and more driving assistance devices. From the early passive driving assistance devices, such as ABS, Air Bag and EBD, the driver can still maintain the ability to control the vehicle during emergency braking or collide with the vehicle. In order to protect the personal safety of the driver, there are still deficiencies. In order to further enhance the driver's driving safety and avoid the driver's own vehicle accidents caused by external vehicle factors, active driving assistance devices have been developed. Warnings are issued in advance in front of the vehicle, such as the existing Lane Departure Warning System (LDWS) or the Forward Collision Warning System (FCWS), which continuously detects the front of the vehicle. The lane line (or sideline), when the car does not deviate from the original driving lane or road, the system will issue an appropriate warning tone to alert the driver, but the disadvantage of the system is that the road must have a clear lane line or standard Line can identify whether the vehicle is off the lane, when the light is weak, the next In the case of snow or fog, the system may not correctly identify the lane line; the Front Collision Warning System (FCWS) detects the distance between the vehicle and the vehicle in front through the image or radar wave, if between the two vehicles. The distance is too close, that is, the brake system that alerts or activates the vehicle to maintain a safe distance or to stop the vehicle in an emergency, to avoid the vehicle chasing or slowing down the collision force, but the disadvantage of the system is that the collision point cannot be predicted in advance, and the vehicle in front needs It is only possible to enter the system detection area (if the vehicles on both sides are unable to operate).

The foregoing active driving assistance device can only alert the driver or stop the vehicle when the vehicle deviates from the lane or is about to collide, and basically still needs the driver to drive the vehicle to the destination, when the above-mentioned active driving assistance device is used for automatic driving of the vehicle. At the time, there are still deficiencies. For example, in addition to detecting road obstacles, automatic driving requires further planning of destination routes and identification of road conditions.

Therefore, the recently developed active driving assistance device, such as the US Patent No. 8195354 "Object detection and classification for autonomous vehicles", is equipped with ranging and image sense on the vehicle. Detector, 3D LiDAR and GPS electronic maps, using ranging and image sensors to identify obstacles moving on the driving path, while 3D radar and GPS electronic maps compare scenes to avoid fixed obstacles. This improves the accuracy of the identification of obstacles around the vehicle, but existing 3D radars can accurately scan terrain or obstacles, but they are expensive and require a large amount of pre-built The database is used to store map data, and there is a problem that the construction cost is too high.

The method and apparatus for combining dual vision images is a method for recognizing a directional light of a preceding vehicle by means of image recognition, and an image processing apparatus and the method thereof. The light signal, and according to the identification result of the direction light signal, predict the direction of travel of the vehicle; however, it does not reveal how to effectively identify the light of the direction light, and the combination of the automatic driving technology of the vehicle itself or the control technology characteristics of the vehicle, There are still shortcomings in applications that combine with autonomous driving.

As mentioned above, the existing active driving assistance device still has deficiencies, and the problem that the automatic driving is expensive and difficult to integrate with the existing system is provided. Therefore, the main object of the present invention is to provide an active automatic driving assistance system and method. Solve the above problems of expensive construction and the like.

The main technical means adopted to achieve the foregoing objective is that the active automatic driving assistance system is built on a vehicle and includes: an autonomous driving control device coupled with the control system of the vehicle for issuing a warning Or controlling the driving state of the vehicle; a road environment detecting device coupled to the autonomous driving control device, the road environment detecting device for detecting the distance of the vehicle in front and identifying the direction light of the vehicle; Capital application device, which is connected with the aforementioned autonomous driving control device The vehicle-mounted application device can be used for transmitting and receiving external signals, planning the vehicle's driving path and vehicle positioning, and a vehicle safety integration device coupled with the autonomous driving control device for detecting the driver. status.

The main technical means adopted to achieve the foregoing objective is to enable the aforementioned active automatic driving assistance method to include: generating a destination planning route; and detecting road environment: obtaining environmental information of a road ahead of the vehicle, the environmental information may include Lane image, distance and direction of the preceding vehicle, and provide collision warning; location detection: obtain the actual position of the vehicle and compare with the driving path to meet the planned route; vehicle safety integration: detection and judgment The state of the driver, actively controlling the driving state of the vehicle when the vehicle is required to be assisted; and the autonomous driving control: determining whether the lane image contains the lane line, and if so, the automatic driving mode according to the planned route and the positioning detection The lane-by-lane line controls the vehicle to travel to the destination. If not, it detects whether the direction of the preceding vehicle is the same as the planned route. If it is judged to be the same as the planned route, it combines the position detection to follow the preceding vehicle to the destination in the automatic driving mode.

The active automatic driving assistance system composed of the foregoing components is integrated with the control system of the vehicle, and the vehicle positioning and comparison driving path is performed by the vehicle-mounted application device, and the vehicle is continuously detected by the road environment detecting device. In the surrounding environment, the autonomous driving control device finds the vehicle with the same direction as the destination of the vehicle, performs automatic follow-up to drive the vehicle to the destination, and recognizes the movement of the preceding vehicle by identifying the signal of the preceding vehicle direction light. Reduce the occurrence of emergency braking and vehicle collision to improve driving efficiency. The vehicle safety integration device detects the driver's state. If necessary (such as the driver's disability), the autonomous driving control device forcibly controls the vehicle to travel, and the vehicle is used by the vehicle. The device sends a signal for assistance. The invention does not need to set up expensive radar equipment and can be integrated with the vehicle control system to achieve complete automatic driving, and solves the problem that the cost of the existing automatic driving is difficult to integrate with the system.

10‧‧‧Autonomous driving control device

20‧‧‧Road environment detection device

21‧‧‧Image capture module

22‧‧‧Car distance sensing module

30‧‧‧Vehicle application device

40‧‧‧Vehicle safety integration device

50‧‧‧Control system

60‧‧‧ Body Signal Module

70‧‧‧ prompt module

80‧‧‧ roads

81, 82‧‧ ‧ road boundaries

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a block diagram of a circuit in accordance with a preferred embodiment of the present invention.

2 is a flow chart of automatic driving in accordance with a preferred embodiment of the present invention.

3 is an image view of a road and road boundary in accordance with a preferred embodiment of the present invention.

4 is a diagram showing the result of identification of a road boundary in accordance with a preferred embodiment of the present invention.

Figure 5 is a schematic illustration of road curvature calculation in accordance with a preferred embodiment of the present invention.

6A, 6B, and 6C are schematic views of a safety distance of a preferred embodiment of the present invention.

7A and 7B are schematic views of a lane change in a preferred embodiment of the present invention.

Figure 8 is a flow chart showing the identification of a directional light lamp in accordance with a preferred embodiment of the present invention.

9 is a flow chart of detecting vehicle collision time and distance in accordance with a preferred embodiment of the present invention.

Figure 10 is a flow chart of the driving safety mechanism of the preferred embodiment of the present invention.

Figure 11 is a schematic diagram (1) of predicting a change of a target position of a lane in accordance with a preferred embodiment of the present invention.

Figure 12 is a schematic diagram (2) of an estimated shift lane target position in accordance with a preferred embodiment of the present invention.

Referring to the preferred embodiment of the present invention, referring to FIG. 1, the active automatic driving assistance system of the present invention is built on a vehicle (not shown), and includes an autonomous driving control device 10 and a road. The environment detecting device 20, a vehicle-mounted application device 30, and a vehicle safety integration device 40 are connected to the road environment detecting device 20, the vehicle-mounted application device 30, and the vehicle safety integration device 40, respectively; The autonomous driving control device 10 is further coupled to a control system 50 of the vehicle and a body signal module 60. The control system 50 controls the throttle and the brake of the vehicle. The body signal module 60 acquires the body signal of the vehicle, such as an acceleration. , steering wheel corner, brake signal or throttle signal.

The autonomous driving control device 10 obtains the vehicle body signal of the vehicle by the body signal module 60, and after the integration operation, an alert module 70 connected thereto is alerted, or the control system 50 actively controls the brake or throttle of the vehicle to control The driving state of the vehicle can cause the vehicle to travel or stop. In the preferred embodiment, the autonomous driving control device 10 is a microcontroller (MCU) or a digital signal processor (DSP), and the control system 50 is An electronic control unit (ECU), the autonomous driving control device 10 can also be further The steps are incorporated in control system 50.

The road environment detecting device 20 detects the distance between the vehicle and the vehicle in front, and identifies the direction light signal at the rear of the vehicle, and includes an image capturing module 21 and a distance sensing module 22. The image module 21 captures the road image in front of the vehicle, and the road environment detecting device 20 identifies the vehicle in front and its direction light signal or an obstacle (such as a road boundary or a guardrail) according to the image. The module 22 detects the distance between the vehicle and the vehicle in front. In the preferred embodiment, the image capturing module 21 is a camera that can be disposed at the front of the vehicle or at the front windshield of the vehicle and faces the vehicle. In front shooting, the distance sensing module 22 is disposed at the front of the vehicle, and uses ultrasonic waves, millimeter waves or 2D Lidar detection.

The in-vehicle application device 30 is configured to send and receive external wireless signals, and is provided with an electronic map to plan a driving path of the vehicle. The wireless signal includes a GPS positioning signal or an AGPS positioning signal, and the in-vehicle application device 30 confirms according to the received coordinates. The location of the vehicle is compared with the driving route set by the electronic map, and the in-vehicle application device 30 can communicate with an external monitoring or ambulance center via wireless transmission to notify the driver or the location and condition of the vehicle.

The vehicle safety integration device 40 is configured to detect the state of the driver, and the driver's driving state is detected by an in-vehicle camera (not shown) or a physiological signal monitoring module (not shown), for example. Drowsiness, coma or drunk driving, and further outward from the vehicle-mounted application device 30 Signal number.

Referring to FIG. 2, the autonomous driving control device 10 performs automatic driving by performing the following steps: generating a destination planning route (201); and detecting road image acquired by the road environment detecting device 20 for the planning path. There is a lane line (or road side line) (202); if it is judged that there is a lane line, the vehicle steering wheel (driving direction) is controlled according to the curvature of the lane line (203); it is judged whether it reaches the destination (204), otherwise it returns to the step (202). If yes, then (205) ends the automatic driving; if the step (202) determines that there is no lane line, the road distance sensing module 22 (2D Lidar) detects the road boundary (or the road edge) (206); When the sensing module 22 can detect the road boundary (or the road edge), the process returns to step (203). If the road boundary (or the road edge) cannot be detected, the image capturing module 21 (camera) detects Measure whether there are other vehicles in front of the vehicle (207); if there is no other vehicle, return to step (202); if there are other vehicles, select a target vehicle and determine whether the target vehicle is the same as the planned route (208); If the direction of the car is different from the planned route, update the target car to re- Given the same target vehicle path planning (209), and returns to step (208); if the target traveling direction of the vehicle the same as the planned path, according to the target vehicle The travel path controls the vehicle steering wheel (direction of travel) (210) and returns to step (204) until the vehicle travels to the destination.

Referring to FIGS. 3 and 4, FIG. 3 is a road image obtained by the image capturing module 21 of the road environment detecting device 20. The road image includes a road surface of the road 80 and two road boundaries 81 and 82, as shown in the figure. The road boundaries 81 and 82 are curved along the road 80 to form an arc. 4 is a road environment detecting device 20 detecting the road boundary 81, 82 of the image of FIG. 3, and the X axis represents the left or right side of the vehicle, and the central point of the X axis (0 meter). Where is the position of the image capturing module 21 corresponding to the vehicle, the right side of the central point of the X axis corresponds to the right side of the vehicle, the left side of the center point corresponds to the left side of the vehicle, and the Y axis direction represents the road extending distance in front of the vehicle The two curves shown in FIG. 4 correspond to the road boundaries 81 and 82 of FIG. 3. It can be seen that the longer curve on the right side corresponds to the road boundary 82 (about 30 meters) on the right side of the vehicle and is curved toward the left side. The short curve corresponds to the road boundary 81 (about 15 meters) on the left side of the vehicle and is slightly curved toward the left side.

Referring to FIG. 5, after the road environment detecting device 20 obtains the identification result of the road boundary 81, 82, the applicant's patent application No. 096145498 "Method and device for detecting vehicle offset" is based on The curvature of the road boundary or the lane line can calculate the rotation angle (rotation angle) of the steering wheel of the vehicle, and the rotation angle tan(α) is calculated by the following formula, L _y = R × sin θ

W _L = R × cos θ

W _R =L _y ×tanα=R×sinθ×tanα

Radius of curvature:

Angle of rotation:

Curb quadratic curve: x=k _w ×y ² +m _w ×y+b _w

Curb curvature:

The distance between the curb and the lane lines on the left and right sides of the vehicle:

W _x = W _L + W _R - △ Y _L The secondary curve of the lane lines on the left and right sides of the vehicle:

x _L = k _w × y ² + m _w × y + (b _{w -} W _x ) where: x, y are the horizontal and vertical axes of the actual space coordinates; k _w , m _w , b _w are the curb times The coefficient of the curve is the information obtained by image detection.

Referring to FIGS. 6A, 6B, 6C and 7A, 7B, the road environment detecting device 20 recognizes the distance between the preceding vehicle and the vehicle, and further determines the risk of moving direction and speed to estimate The possible collision distance and time of the vehicle, as shown in FIGS. 6A and 6B, the current vehicle is cut into the front lane of the vehicle by other (left or right) lanes, and when the safety distance is insufficient, the road environment detecting device 20 determines The direction of the directional light, and the self-driving control device 10 causes the prompting module 70 to issue a warning, and the braking system is timely generated by the control system 50 to avoid collision; as shown in FIG. 6C, the current vehicle is the opposite lane. To turn left and cut into the car In the forward lane, the autonomous driving control device 10 causes the prompting module 70 to immediately issue an alert and generate a braking action through the control system 50 to avoid a collision. As shown in FIGS. 7A and 7B, when there is sufficient space (Free Space) or a vehicle distance for switching lanes in the front left or right lane of the vehicle, if the preceding vehicle temporarily changes lanes, the autonomous driving control device 10 transmits the control system. 50 Reduce the number of vehicle brakes by changing lanes, which can improve driving efficiency and save fuel consumption.

Referring to FIG. 8 , the road environment detecting device 20 recognizes the light of the preceding vehicle direction light to perform the following steps: acquiring the front image of the vehicle (801); detecting the vehicle (802) in the image; determining the image Whether there is a vehicle present (803), if there is no vehicle, return to step (801); if there is a vehicle in the image, estimate the vehicle position and size (including height and width) (804); set the ROI of the image (805) Binarization (threshold Th_Y, Th_U, and Th_V) (806); subtraction (807) centered on the symmetry axis of the vehicle; performing type filtering (erosion or expansion) (808); estimating the center point Position (809); judge left or right (810); determine whether there are n consecutive frames (Frame) and disappear n frames (Frame) (811), if otherwise, return to step (801), and if so, it is determined that the left or right direction light of the preceding vehicle has a light signal (812).

Since the direction light of the existing vehicle is orange, light yellow or red, the threshold Th_Y of step (806) can be set to one of values between 235 and 250, preferably 246, and the threshold Th_U can be set to 95~ One of the values between 115, preferably 108, the threshold Th_V can be set to one of values between 120 and 140, preferably 130. Moreover, since the directional light of the vehicle is bilaterally symmetrical, the vehicle will only illuminate one side each time the lane is changed, and if the two sides illuminate at the same time, it is a vehicle fault warning, so it is only necessary to recognize the directional light signal on one side. Furthermore, the blinking frequency of the existing directional light is 1 to 2 times per second, so the number of the aforementioned frames can be calculated correspondingly.

Referring to FIG. 9 , the road environment detecting device 20 estimates the vehicle collision distance and time by performing the following steps: detecting the surrounding environment of the vehicle by the distance detecting module 22 ( 901 ); identifying by the image capturing module 21 The obstacle type (902); determining whether the obstacle is a vehicle (903), if the obstacle is not a vehicle, returning to step (901); if the obstacle is a vehicle, determining that the direction light is left turn Or right turn (904); predict the collision time by the collision prevention algorithm after determining the direction light (905); start the driving safety mechanism through the autonomous driving control device 10 System 50 controls the vehicle (906).

Referring to FIG. 10, the driving safety mechanism described above is performed by the autonomous driving control device 10: determining whether the vehicle is detected in the front (1001); if the front is a vehicle and the collision time is less than the threshold value of the braking (1002), Start the brake assist system (1003) immediately; if the collision time is greater than the threshold value of the brake, determine whether there is enough dodge space around the vehicle (1004); if there is enough dodge space in front of the vehicle, the driver is prompted or the steering wheel is involved. To move the vehicle to the dodge space (1005); if there is not enough space in front of the car, an alert is immediately issued to alert the driver to the dangerous situation (1006).

Referring to FIGS. 11 and 12, the above estimated lane change target position is calculated by the following formula. Where (gx, gy) is the target position after the predicted lane change, x is the lane width, and y is the longitudinal distance of the predicted movement. For predicting the direction of travel. As shown in FIG. 12, the predicted traveling direction of the preceding vehicle is obtained according to the direction light identification ( ) and merge with the direction of travel (φ) predicted by the tracking algorithm to get the final direction of travel =φ± , the estimated direction of travel of the vehicle ahead ( Brought into the formula as well as The predicted collision time can be obtained, and the accuracy of the prediction is increased.

10‧‧‧Autonomous driving control device

20‧‧‧Road environment detection device

21‧‧‧Image capture module

22‧‧‧Car distance sensing module

30‧‧‧Vehicle application device

40‧‧‧Vehicle safety integration device

50‧‧‧Control system

60‧‧‧ Body Signal Module

70‧‧‧ prompt module

Claims (9)

An active automatic driving assistance method includes: generating a destination planning route; and detecting road environment: obtaining environmental information of a road ahead of the vehicle, the environmental information may include a lane image, a vehicle distance, and a direction signal of the preceding vehicle And providing collision warning; positioning detection: obtaining the actual position of the vehicle and comparing with the driving path to meet the planned route; vehicle safety integration: detecting and judging the state of the driver, when assisting driving the vehicle, Actively controlling the driving state of the vehicle; and autonomous driving control: determining whether the lane image contains the lane line, and if so, controlling the vehicle to the destination according to the planned route and the positioning detection by the automatic driving mode, if none Then, it is detected whether the driving direction of the preceding vehicle is the same as the planned route. If it is judged to be the same as the planned route, the positioning detection is combined with the preceding driving to the destination in an automatic driving manner. The active automatic driving assistance method according to claim 1 obtains the identification result of the lane line, and calculates the rotation angle of the steering wheel of the vehicle according to the curvature of the lane line, and the rotation angle is calculated by the following formula: x=k _w × y ² + m _w × y + b _w , curb curvature: , radius of curvature: , steering wheel angle: L _y = R × sin θ , Where x and y are the horizontal and vertical axes of the actual space coordinates, respectively; k _w , m _w , b _w are the coefficients of the quadratic curve of the curb. The active automatic driving assistance method according to claim 2, wherein the identification light of the direction light obtains a front image of the vehicle itself, and detects the image. The vehicle is estimated and its position and size are set, ROI, binarization, left and right subtraction centered on the vehicle symmetry axis, type filtering (erosion or expansion), estimation of the center point position and judgment of the left or right direction lights, and judgment Whether there are n frames in succession and disappears n frames, and if so, it is judged that the left or right direction lights of the preceding vehicle have a light. The active automatic driving assistance method according to claim 3, wherein the estimated vehicle collision distance and time is to detect a surrounding environment of the vehicle, and the light of the direction light of the rear of the vehicle is determined to be a left turn or a right turn, and the direction light is judged. The collision time is predicted by the collision prevention algorithm to start the driving safety mechanism to control the vehicle. The active automatic driving assistance method according to claim 4, wherein the driving safety mechanism is to detect whether the collision time of the preceding vehicle is less than the threshold value of the braking vehicle, and if the collision time is less than the threshold value of the braking vehicle, the braking assistance system is activated, and the collision time is The threshold value greater than the brake is to determine whether there is enough doping space around the vehicle. If there is enough doping space, the driver is prompted or the steering wheel control is involved to move the vehicle to the dodge space. The active automatic driving assistance method according to claim 5 detects and judges the state of the driver, and when the driver loses the driving ability, the autonomous driving control forcibly intervenes to control the vehicle to travel. An active automatic driving assistance system is built on a vehicle and includes: an autonomous driving control device coupled with a control system of the vehicle for issuing a warning or controlling the driving state of the vehicle; a detection device coupled to the autonomous driving control device, wherein the road environment detecting device is configured to detect a vehicle distance of a preceding vehicle and identify a directional light of the vehicle; an in-vehicle application device coupled to the autonomous driving control device, the vehicle-mounted application device capable of transmitting and receiving external signals, planning a vehicle's driving path and vehicle positioning; and a vehicle safety integration The device is coupled to the autonomous driving control device for detecting a state of the driver; thereby, the road environment detecting device determines whether the lane image contains a lane line, and if so, the vehicle The capital application device controls the vehicle to travel to the destination according to the planning route and the positioning detection for the autonomous driving control device to drive the vehicle to the destination by the automatic driving mode. If not, the road environment detecting device detects whether the driving direction of the preceding vehicle is planned and planned. Similarly, if it is judged to be the same as the planned route, combined with the position detection to follow the preceding vehicle to the destination in an automatic driving manner. The active automatic driving assistance system according to claim 7, wherein the autonomous driving control device is coupled to a body signal module, and is integrated with the body signal and is alerted by a prompting module connected thereto or through the control system. Actively control the brakes or throttle to make the vehicle travel or stop. The active environment driving detection device according to claim 7 or 8, wherein the road environment detecting device comprises an image capturing module and a distance sensing module, wherein the image capturing module obtains an image of the front of the vehicle itself and is The road environment detecting device identifies the vehicle in front and its direction light signal, and the distance sensing module detects the distance between the vehicle and the vehicle in front.