KR20220140087A - Apparatus and method for danger recognition and accident evasion using motion of forward vehicle - Google Patents

Abstract

The present invention relates to a device and a method for recognizing a danger and controlling a vehicle to identify a dangerous situation based on a motion of a forward vehicle and prevent an accident for preventing the accident of an autonomous driving vehicle due to an object or a condition which cannot be recognized by an existing sensor. First, the device extracts motion information of the forward vehicle from state information of the forward vehicle obtained by sensors such as a camera, a radar, and a lidar or received by V2X communication or communication with traffic control. The motion information relates to deceleration, disappearance, appearance, pitch, roll, and yaw of the forward vehicle. The device estimates the dangerous situation of a current driving road from the extracted motion information of the forward vehicle. When the device extracts the motion information of the forward vehicle to estimate the dangerous situation, the device controls the vehicle to correspond to an estimated situation.

Description

{Apparatus and method for danger recognition and accident evasion using motion of forward vehicle}

The technical field of the present invention relates to environmental sensing, image processing, vehicle control, and autonomous driving technology.

In the current sensor system of a car, it is possible to recognize a general object such as a vehicle in front, a pedestrian, etc., and to avoid accidents such as a collision by predicting and judging the risk of an accident with the recognized object. However, atypical objects such as falling objects on the road, small obstacles, and bumps that are not defined on the precise map may not be recognized by the vehicle sensor, especially in the case of an autonomous vehicle that the driver does not directly drive. It is difficult to adequately deal with accident risk situations caused by

Korean Patent Publication No. 10-2019-0059567; Release date May 31, 2019

Since the cognitive ability of an autonomous vehicle is dependent on sensors and precision maps, it can properly respond to dangerous situations that can be detected by sensors or precision maps, but it is difficult to properly respond to accident risks in atypical objects or sudden danger situations that are not detected by sensors.

Accordingly, the present invention proposes a risk determination and vehicle control method for identifying a dangerous situation based on the motion of a front vehicle and preventing an accident in order to prevent an accident of an autonomous vehicle caused by an object that cannot be recognized by a sensor.

The concept of the front vehicle motion-based risk recognition and accident avoidance device and method of the present invention recognizes the state information of the front vehicle and surrounding obstacles (objects) of the own vehicle with sensors such as cameras, radars, lidars, or V2X (V2V, V2V, V2I, etc.) Based on the motion information extracted from the status information of the vehicle in front obtained through communication or communication with traffic control, it determines the dangerous situation ahead and controls the own vehicle accordingly (control of speed and/or steering) will do

Specifically, the motion information of the front vehicle is extracted from the status information of the vehicle in front acquired by a sensor or through communication with V2X or traffic control. Motion information includes the speed, acceleration, deceleration, disappearance, appearance, pitch, roll, and yaw of the vehicle ahead. For example, when the size of the vehicle in front increases in the image acquired by the camera, the distance to the vehicle in front increases. In this case, the motion information of the vehicle in front is 'deceleration'. The deceleration rate (deceleration rate) of the vehicle in front can be estimated by the rate of change at which the size of the vehicle in front increases. And the 'disappearing and appearing' of the vehicle in front includes not only the disappearance/appearance of the entire vehicle body but also the disappearance/appearance of a part of the vehicle body. 'Pitch, roll, yaw' is the shaking of the vehicle, and when the vehicle in front moves over a speed bump or avoids an obstacle (e.g., an unattended object or a falling object), pitch, roll, yaw will happen Pitch, roll, and yaw motion information can be extracted from the image by changing the shape of the vehicle ahead. For example, the pitch can be determined by increasing or decreasing the area of the rear surface and upper and lower surfaces of the vehicle body in the front vehicle object, and the roll can be determined by moving the left and right sides of the rear shape of the front vehicle body up and down, It can be determined that the area of the left side and the right side of the front vehicle body increases or decreases compared to the rear side. Such image processing can be processed based on artificial intelligence (machine learning, deep learning, CNN, DNN, etc.). Also, as another example, in addition to the camera image, motion information of the vehicle in front can be estimated by measuring with a sensor such as radar or lidar, or movement information can be received from control infrastructure or other vehicles using V2X or communication with traffic control. have.

From the motion information of the vehicle in front extracted in this way, it is possible to estimate a dangerous situation on the current driving road. For example, when the motion information of the vehicle in front is deceleration, the possibility of collision with the vehicle in front occurs according to the deceleration, which becomes a dangerous situation. The disappearance and appearance of the vehicle in front indicates that the road ahead has a dangerous situation such as a cliff, a sharp turn, or a mountain road. The disappearance and appearance of the vehicle in front includes not only the disappearance/appearance of the entire vehicle body but also the disappearance/appearance of a part of the vehicle body. This motion information can be extracted by various methods as mentioned above. Pitch, roll, and yaw are vibrations of the vehicle, which are caused by the vehicle in front crossing a speed bump, quickly avoiding an obstacle (e.g., an unattended object or falling object), or sliding on an icy road due to low friction. Pitches, rolls, and yogas will occur during the motion of the back.

When a dangerous situation is estimated by extracting the motion information of the vehicle ahead as described above, the vehicle is controlled according to the estimated situation. For example, when a situation such as a speed bump, ice sheet, or a cliff is estimated, the vehicle is decelerated, and when a situation such as a forward falling object (obstacle) or a mountain road is estimated, deceleration and/or steering control is performed.

The configuration and operation of the present invention will become clearer through specific embodiments described later in conjunction with the drawings.

According to the present invention, it is possible to prevent accidents by predicting dangerous situations such as collisions and falls through driving information such as motion of the front vehicle in the above-mentioned dangerous situations, and in particular, various atypical objects or imminent collisions that may occur on the road Able to deal effectively with the situation.

1 is a conceptual block diagram of a front vehicle motion-based risk recognition and accident avoidance device and method according to the present invention;
2 is a block diagram of a preferred embodiment of the present invention shown in FIG.
3 is a schematic diagram to help understand pitch, roll, and yaw;
Fig. 4 is a block diagram of a modified embodiment of the embodiment of Fig. 2;
5 is a configuration diagram of an embodiment in which information about surrounding objects recognized by the sensor unit 100 is taken into account.
6 is a processing flowchart of a risk recognition and accident avoidance method based on a front vehicle motion
7 is a processing flowchart of a modified embodiment of the forward vehicle motion-based risk recognition and accident avoidance method of FIG.

Advantages and features of the present invention, and methods of achieving them, will become apparent with reference to the preferred embodiments described in detail below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments described below and may be implemented in various other forms. The examples are merely provided to completely disclose the present invention and to completely inform those of ordinary skill in the art to which the present invention pertains to the scope of the invention, and the present invention is defined by the claims will be. In addition, the terminology used herein is for the purpose of describing the embodiment and is not intended to limit the present invention. In this specification, the singular also includes the plural unless otherwise specified. Also, as used herein, the term 'comprises (comprises, comprising, etc.)' refers to one or more elements, steps, operations other than the components, steps, operations, and/or elements (or parts) referred to by XX. , and/or is used in the sense that it does not exclude the presence or addition of elements (or components).

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the embodiment, if a detailed description of a related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

1 is a basic configuration diagram of an embodiment of a front vehicle motion-based risk recognition and accident avoidance apparatus and method according to the present invention. The embodiment shown in FIG. 1 shows an embodiment in which a camera is used as a sensor to recognize a vehicle in front of the host vehicle and an obstacle (object) around it. However, as mentioned above, in the present invention, recognition of the front vehicle and surrounding obstacles (objects) using radar, lidar, etc. other than the camera, and acquisition using V2X communication or communication with traffic control is also possible. In the following embodiment, image acquisition and image processing using a camera will be mainly described.

Referring to FIG. 1 , the present embodiment mainly includes a sensor unit 100 for recognizing a front vehicle and surrounding obstacles (objects) of the host vehicle; a dangerous situation determination unit 200 for determining a forward dangerous situation based on the motion information of the vehicle in front recognized by the sensor unit 100; It is composed of a vehicle control unit 300 that controls the speed and/or steering of the own vehicle based on the motion of the front vehicle estimated by the dangerous situation determination unit 200 . Here, the vehicle control unit 300 may take into account information about surrounding objects recognized by the sensor unit 100 in addition to the motion of the vehicle in front when controlling the vehicle.

The sensor unit 100 detects surrounding vehicles and objects (including obstacles such as falling objects and unattended objects on the road) including the front vehicle, and may be implemented as an environmental sensor such as a camera, radar, and lidar. The sensor unit 100 may be composed of individual sensors or a fusion of sensors. Information on the front vehicle and surrounding obstacles detected by the sensor unit 100 includes relative position relative to the own vehicle, speed, acceleration, deceleration, and motion information such as yaw, roll, and pitch.

A detailed description of the dangerous situation determination unit 200 and the vehicle control unit 300 will be described below with reference to FIG. 2 .

2 is a block diagram of a preferred embodiment of the present invention shown in FIG.

The dangerous situation determination unit 200 includes a front vehicle motion information extraction unit 210 that receives an image or measurement information (hereinafter, collectively referred to as image information) from the sensor unit 100 and extracts motion information of the vehicle in front; and a dangerous situation estimator 220 for determining a forward dangerous situation based on the motion information.

3 is to help understand pitch, roll, and yaw. The rotation about the axis X transversely dividing the center of the vehicle, the axis Y dividing the longitudinal direction, and the axis Z dividing the vertical direction are defined as pitch, roll, and yaw, respectively.

Returning to FIG. 2 again, the dangerous situation estimating unit 220 estimates the dangerous situation of the current driving road from the motion information of the front vehicle extracted by the forward vehicle motion information extraction unit 210 . For example, if the motion information of the vehicle in front is deceleration, the risk of collision with the vehicle in front is estimated according to the deceleration, and the disappearance and appearance of the vehicle in front causes the risk of a cliff, a sharp turn, a mountain road, etc. inform you that there is It can be estimated that there is a speed bump in front (ie, in front of the vehicle in front) through a change in pitch by a certain amount or more and a decrease in driving speed. Due to a sudden change in lateral motion (roll and/or yaw), it can be estimated that the vehicle in front avoids an obstacle or a falling object, or a low friction situation such as ice. Atypical environments such as mountain roads and cliffs can be estimated through the sudden disappearance of the vehicle in front or a sudden change in motion.

The vehicle control unit 300 includes a dangerous situation classification unit 310 that classifies the vehicle control into a control category to cope with the dangerous situation estimated by the dangerous situation determination unit 200, and controls the vehicle according to the classified dangerous situation. and a control signal generator 320 for generating a control signal for With the generated control signal, the vehicle controller 300 controls the speed (acceleration, deceleration, braking) and steering of the own vehicle to avoid an accident. As some examples, when a front speed bump, a sharp turning road, a mountain road, a cliff, etc. are detected, the speed of the own vehicle is reduced (rapidly decelerated depending on the situation). If there is ice or an obstacle is presumed to be present, decelerate and/or steer to change lanes to an adjacent lane or stop the vehicle.

Fig. 4 shows a modified embodiment of the embodiment of Fig. 2; In the embodiment of FIG. 2 , when the dangerous situation determination unit 200 estimates a dangerous situation, this estimation information is transmitted to the vehicle control unit 300 and the dangerous situation is classified by the dangerous situation classification unit 310 , but in the embodiment of FIG. In the example, the dangerous situation classification unit 230 is included in the dangerous situation determination unit 200 so that the dangerous situation classification information is directly transmitted to the vehicle control unit 300 , and the vehicle control unit 300 generates a corresponding control signal to generate a corresponding control signal to the own vehicle. to control

In addition, the present invention may be modified in various embodiments.

5 is a configuration diagram of an embodiment in which information about surrounding objects recognized from the sensor unit 100 in addition to the motion of the vehicle in front is taken into account when determining a dangerous situation as described above.

2, in addition to the front vehicle motion extraction unit 210 that extracts the motion of the front vehicle from the image from the sensor unit 100 to the dangerous situation determination unit 200, object information such as surrounding obstacles is extracted. An object extraction unit 240 is included. In this way, by determining a dangerous situation based on the surrounding object information as well as the motion information of the vehicle in front, the vehicle control unit 300 can perform more sophisticated and suitable vehicle control than the case of the embodiment of FIG. 2 or FIG. 3 . It is possible to maximize the achievement of the purpose of risk recognition and accident avoidance.

6 is a processing flowchart of an embodiment of a method for risk recognition and accident avoidance based on a front vehicle motion performed in the apparatus of the present invention as described above. The embodiment shown in Fig. 6 (and Fig. 7 below) also shows an embodiment when a camera is used as a sensor to recognize a vehicle in front of the host vehicle and surrounding obstacles (objects), but radar, lidar, etc. other than the camera are shown. Recognition of the used front vehicle and surrounding obstacles (objects), and the possibility of obtaining using V2X communication or communication with traffic control are the same as mentioned above.

First, (eg, the sensor unit 100 of FIG. 1 ) acquires a front image including the vehicle in front ( 410 ).

(For example, the dangerous situation determination unit 200) extracts motion information of the front vehicle from the image information (received from the sensor unit 100) (420). Here, the motion information includes motion information such as a position relative to the host vehicle, speed, acceleration, deceleration, or yaw, roll, and pitch.

(For example, the dangerous situation determination unit 200) estimates a forward dangerous situation from the extracted motion information (430). For example, it is estimated that there is a speed bump in front by using a certain amount of pitch change and driving speed of the vehicle in front, or obstacles such as objects falling in front or left on the road through a sudden change in the yaw of the vehicle in front. Estimate the avoidance situation or low friction situation such as ice. In addition, atypical situations such as sharp turns, mountain roads, cliffs, and falling rocks are estimated through the sudden disappearance/appearance of the vehicle ahead or a sudden change in motion.

(For example, the dangerous situation determination unit 200 or the vehicle control unit 300) classifies the estimated dangerous situation into a control category for vehicle control to cope with the estimated dangerous situation (440).

And (for example, the vehicle control unit 300) generates a control signal for controlling the vehicle according to the classified dangerous situation (450).

Finally (eg, the vehicle controller 300) controls the speed (acceleration, deceleration, braking) and steering of the host vehicle using the generated control signal to avoid an accident ( 460 ).

7 is a processing flowchart of a modified embodiment of the forward vehicle motion-based risk recognition and accident avoidance method of FIG. 6 . Fig. 7 is a flow chart of a method corresponding to the apparatus shown in Fig. 5; The difference from the flowchart of FIG. 6 is that, in addition to extracting 420 the motion information of the vehicle ahead from the image information, extracting (425) object information from the image information is additionally included. In the step of estimating the dangerous situation 430, the dangerous situation is estimated based on both the extracted front vehicle motion information and the object information.

In this way, by judging a dangerous situation based on the motion of the vehicle ahead and surrounding objects, more sophisticated and appropriate vehicle control can be performed, thereby maximizing the purpose of risk recognition and accident avoidance of the present invention.

Although the present invention has been described in detail through preferred embodiments of the present invention, those of ordinary skill in the art to which the present invention pertains will realize that the present invention is different from the contents disclosed in the present specification without changing the technical spirit or essential features thereof. It will be understood that the invention may be embodied in other specific forms. It should be understood that the embodiments described above are illustrative in all respects and not restrictive. In addition, the protection scope of the present invention is determined by the claims described later rather than the detailed description, and all changes or modifications derived from the claims and their equivalent concepts should be interpreted as being included in the technical scope of the present invention. do.

Claims (16)

extracting motion information of the front vehicle from the status information of the front vehicle;
estimating a dangerous situation on the current driving road from the extracted motion information of the vehicle ahead;
A front vehicle motion-based risk recognition and accident avoidance device, configured to control a vehicle according to a dangerous situation estimated from the extracted front vehicle motion information. The method of claim 1, wherein the extracted motion information of the vehicle ahead is
A front vehicle motion-based risk recognition and accident avoidance device comprising at least one of speed, acceleration, deceleration, disappearance, appearance, pitch, roll, and yaw of the vehicle ahead. The method of claim 1, wherein the control of the vehicle comprises:
A front vehicle motion-based risk recognition and accident avoidance device comprising at least one of vehicle deceleration, vehicle braking, and vehicle steering. an acquisition unit configured to acquire state information of a vehicle in front of the host vehicle and surrounding objects;
a dangerous situation determination unit for judging a forward dangerous situation based on the forward vehicle motion information extracted from the forward vehicle state information obtained from the acquisition unit; and
A front vehicle motion-based risk recognition and accident avoidance device, comprising a vehicle control unit for controlling the own vehicle based on the motion of the front vehicle estimated from the dangerous situation determination unit. The method of claim 4, wherein the acquisition unit
A front vehicle motion-based risk recognition and accident avoidance device comprising at least one of a camera, a radar, and a lidar that detects state information of a surrounding object including a front vehicle. The method of claim 4, wherein the acquisition unit
A front vehicle motion-based risk recognition and accident avoidance device, including at least one of V2X communication and communication with traffic control for receiving the state of surrounding objects including the vehicle in front. The method of claim 4, wherein the front vehicle motion information
A front vehicle motion-based risk recognition and accident avoidance device comprising at least one of speed, acceleration, deceleration, disappearance, appearance, pitch, roll, and yaw of the vehicle ahead. According to claim 4, wherein the dangerous situation determination unit
Front vehicle motion-based risk recognition, comprising: a front vehicle motion information extraction unit for receiving state information from the acquisition unit to extract motion information of the vehicle in front; and accident avoidance devices. According to claim 4, wherein the dangerous situation determination unit
A front vehicle motion information extraction unit for receiving the state information from the acquisition unit and extracting motion information of the vehicle in front; A front vehicle motion-based risk recognition and accident avoidance device, including a dangerous situation classification unit for classifying into control categories for vehicle control. According to claim 4, wherein the vehicle control unit
A dangerous situation classification unit for classifying into control categories for vehicle control to cope with the dangerous situation estimated by the dangerous situation determination unit, and a control signal generation unit for generating a control signal for controlling the vehicle according to the classified dangerous situation Including, forward vehicle motion-based risk recognition and accident avoidance device. According to claim 4, wherein the vehicle control unit
A front vehicle motion-based risk recognition and accident avoidance device comprising a control signal generator for generating a control signal for controlling the vehicle in response to the dangerous situation estimated by the risk situation determination unit. 5. The method of claim 4, wherein the vehicle control performed by the vehicle control unit
A front vehicle motion-based risk recognition and accident avoidance device comprising at least one of vehicle deceleration, vehicle braking, and vehicle steering. 5. The method of claim 4,
the dangerous situation determination unit is configured to determine a forward dangerous situation based on surrounding object information in addition to the forward vehicle motion information extracted from the forward vehicle state information;
The vehicle control unit is a front vehicle motion-based risk recognition and accident avoidance device, characterized in that configured to perform vehicle control based on the front vehicle motion information and the surrounding object information. A method performed in an electronic computing device composed of at least one of hardware and software,
acquiring status information including the vehicle ahead;
extracting motion information of the vehicle ahead from the state information;
estimating a dangerous situation ahead from the extracted motion information;
A front vehicle motion-based risk recognition and accident avoidance method, comprising generating a control signal for controlling the vehicle in response to the estimated dangerous situation and controlling the own vehicle with the generated control signal. 15. The method of claim 14, Between the step of estimating the dangerous situation and the step of generating the control signal, further comprising classifying the estimated dangerous situation into a control category for vehicle control, forward vehicle motion-based risk recognition and How to avoid accidents. 15. The method of claim 14,
The step of extracting the motion information of the front vehicle from the acquired state information further includes extracting object information from the state information,
The step of estimating the dangerous situation ahead from the extracted motion information includes estimating the dangerous situation based on both the extracted forward vehicle motion information and the object information. .

Images