CN111845554A - Pedestrian collision early warning method and device based on binocular stereo camera - Google Patents
Pedestrian collision early warning method and device based on binocular stereo camera Download PDFInfo
- Publication number
- CN111845554A CN111845554A CN202010497044.5A CN202010497044A CN111845554A CN 111845554 A CN111845554 A CN 111845554A CN 202010497044 A CN202010497044 A CN 202010497044A CN 111845554 A CN111845554 A CN 111845554A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- judging whether
- velocity vector
- vehicle
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
- B60Q9/008—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling for anti-collision purposes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
- B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
- B60W30/095—Predicting travel path or likelihood of collision
- B60W30/0956—Predicting travel path or likelihood of collision the prediction being responsive to traffic or environmental parameters
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Abstract
The invention discloses a pedestrian collision early warning method and device based on binocular stereo cameras, which can perform early warning when a collision risk exists between a vehicle and a pedestrian. The method comprises the following steps: acquiring a target pedestrian based on a disparity map of a binocular camera; judging whether the target pedestrian is static or not, if so, executing static operation and judging whether to give an early warning or not; otherwise, executing non-static operation: respectively calculating the motion speed vectors of the own vehicle and the pedestrian based on the position change; judging whether the included angle between the two vectors is equal to 90 degrees or not, if so, executing the crossing operation of the pedestrian and judging whether to give an early warning or not; otherwise, performing a pedestrian non-crossing operation: judging whether the included angle between the two vectors is equal to 0 degree or 180 degrees, if so, executing the operation of equidirectional or opposite movement, and judging whether to give an early warning; otherwise, executing the inclined penetrating operation and judging whether to give an early warning.
Description
Technical Field
The invention relates to an advanced driving assistance system, in particular to forward collision early warning based on a binocular stereo camera.
Background
With the popularization of automobiles in ordinary families, traffic accidents are greatly increased, and accidents of casualties and property loss are frequent. The ADAS (Advanced Driving assistance System) can judge possible dangerous conditions in advance and give an alarm, so that traffic accidents are effectively avoided.
There are two technical routes of monocular and binocular in the automobile vision ADAS scheme. The monocular camera for recognizing the obstacles depends on a comprehensive sample base and an excellent classifier, and in a complex and changeable environment, too many unknown obstacles cannot be recognized; the binocular camera can identify various obstacles including various vehicles (special-shaped vehicles, trucks, tricycles, electric vehicles and bicycles), pedestrians and special obstacles without large-scale data acquisition to help machine learning, can construct a three-dimensional space scene in front of a driving road of the vehicle in real time, calculates three-dimensional geometric information of the obstacles and relative distance between the obstacles and a self vehicle, sends alarm prompt to a driver before collision danger possibly occurs, helps the driver to prevent traffic accidents such as collision, rear-end collision and the like caused by various conditions such as fatigue driving, distraction, new hands on the road and the like, and improves driving experience.
Forward Collision Warning (FCW) is an important component of ADAS. The FCW means that a front obstacle is detected in real time through a binocular camera, the relative distance, the direction, the relative speed and the relative acceleration between a vehicle and the obstacle are judged, the Time To Collision (TTC) and the Enhanced Time To Collision (ETTC) are calculated and are respectively compared with a threshold value, the TTC and the ETTC are smaller than or equal to the threshold value, an alarm is given out, and a driver is reminded to take necessary braking and decelerating measures.
Disclosure of Invention
The invention aims to provide a pedestrian collision early warning method and device based on a binocular stereo camera, which can perform collision early warning when a vehicle and a pedestrian in front have collision risk and remind a driver to take necessary braking measures.
The pedestrian collision early warning method based on the binocular stereo camera comprises the following steps:
acquiring a target pedestrian based on a disparity map of a binocular camera; judging whether the target pedestrian is static: if yes, executing static operation, and judging whether forward collision early warning is performed or not; if not, executing non-static operation; the non-stationary operation comprises the steps of: respectively calculating a vehicle motion speed vector and a pedestrian motion speed vector based on the position change; judging whether the included angle between the motion velocity vector of the self vehicle and the motion velocity vector of the pedestrian is equal to 90 degrees or not: if yes, executing pedestrian crossing operation, and judging whether forward collision early warning is carried out or not; if not, executing the pedestrian non-crossing operation; the pedestrian non-crossing operation comprises the steps of: judging whether the included angle between the motion velocity vector of the self vehicle and the motion velocity vector of the pedestrian is equal to 0 degree or 180 degrees: if yes, executing the operation of the same-direction or opposite-direction movement of the self-vehicle and the pedestrian, and judging whether to carry out forward collision early warning; if not, the pedestrian obliquely-passing operation is executed, and whether forward collision early warning is carried out or not is judged.
Wherein the stationary operation comprises the steps of: calculating collision time and reinforced distance collision time, and selecting a smaller value; and judging whether the smaller value is smaller than an alarm threshold value, if so, carrying out forward collision early warning, judging that the collision risk exists, and reminding a driver to take necessary braking measures.
Wherein the pedestrian crossing operation comprises the steps of: calculating collision time and reinforced distance collision time, selecting a smaller value, and calculating pedestrian crossing time, wherein the longitudinal relative speed is equal to the magnitude of the motion speed vector of the vehicle and is used for calculating the collision time and the reinforced distance collision time; the size of the pedestrian motion velocity vector is used for calculating the pedestrian crossing time; judging whether the smaller value is smaller than the crossing time of the pedestrian, if not, not giving an alarm; if yes, continuing to execute the following steps; and judging whether the smaller value is smaller than an alarm threshold value, and if so, alarming.
Wherein the operation of the co-directional or opposite movement of the self-vehicle and the pedestrian comprises the following steps: calculating collision time and reinforced distance collision time, and selecting a smaller value, wherein an included angle between a vehicle motion speed vector and a pedestrian motion speed vector is equal to 0 degree, the vehicle motion speed vector and the pedestrian motion speed vector are judged to move in the same direction and 180 degrees, the vehicle motion speed vector and the pedestrian motion speed vector are judged to move in opposite directions, and the relative speed of the vehicle and the pedestrian is equal to the difference between the vehicle motion speed vector and the pedestrian motion speed vector during the motion in the same direction; when the vehicle moves in the opposite direction, the sum of the magnitudes of the motion velocity vector of the vehicle and the motion velocity vector of the pedestrian is equal to; judging whether the smaller value is smaller than an alarm threshold value, if not, not alarming; if yes, alarming.
Wherein the pedestrian diagonal crossing operation comprises the steps of: calculating collision time and reinforced distance collision time, selecting a smaller value, and calculating pedestrian crossing time, wherein a pedestrian movement velocity vector is decomposed along the direction of a vehicle movement velocity vector and the direction perpendicular to the direction of the vehicle movement velocity vector to obtain a first component vector and a second component vector, the size of the first component vector is used for calculating the collision time and the reinforced distance collision time, and the size of the second component vector is used for calculating the pedestrian crossing time; judging whether the smaller value is smaller than the crossing time, if not, not giving an alarm; if yes, continuing to execute the following steps; and judging whether the smaller value is smaller than an alarm threshold value, and if so, alarming.
The method for acquiring the target pedestrian based on the disparity map of the binocular camera comprises the following steps: detecting an obstacle based on the disparity map; judging whether the barrier is in the alarm area, if so, continuing the following steps; judging whether the barrier is a pedestrian, if so, continuing the following steps; and acquiring the pedestrian with the closest distance as the target pedestrian.
The method comprises the steps of judging whether a target pedestrian is static or not, comparing whether the coordinate of the central position of the target pedestrian is changed or not at the current moment with the coordinate of the current moment when a host vehicle coordinate system at the previous moment is a world coordinate system or not, judging whether the target pedestrian is static or not, judging whether the target pedestrian moves or not, and judging whether the target pedestrian is static or not; the calculated self-vehicle motion velocity vector and the pedestrian motion velocity vector are calculated according to the current self-vehicle central position coordinate and the last self-vehicle central position coordinate to obtain the self-vehicle motion velocity vector, and the pedestrian motion velocity vector is calculated according to the current target pedestrian central position coordinate and the last target pedestrian central position coordinate.
Before judging whether the target pedestrian is static, judging whether the tracking frame number is greater than a preset threshold value, and if not, returning to the initial step; if yes, the following steps are continued.
The present invention also provides a memory device having stored therein a plurality of instructions adapted to be loaded and executed by a processor to: acquiring a target pedestrian based on a disparity map of a binocular camera; judging whether the target pedestrian is static: if yes, executing static operation, and judging whether forward collision early warning is performed or not; if not, executing non-static operation; the non-stationary operation comprises the steps of: respectively calculating a vehicle motion speed vector and a pedestrian motion speed vector based on the position change; judging whether the included angle between the motion velocity vector of the self vehicle and the motion velocity vector of the pedestrian is equal to 90 degrees or not: if yes, executing pedestrian crossing operation, and judging whether forward collision early warning is carried out or not; if not, executing the pedestrian non-crossing operation; the pedestrian non-crossing operation comprises the steps of: judging whether the included angle between the motion velocity vector of the self vehicle and the motion velocity vector of the pedestrian is equal to 0 degree or 180 degrees: if yes, executing the operation of the same-direction or opposite-direction movement of the self-vehicle and the pedestrian, and judging whether to carry out forward collision early warning; if not, the pedestrian obliquely-passing operation is executed, and whether forward collision early warning is carried out or not is judged.
The present invention also provides an automobile with a binocular stereo camera, having: a processor adapted to implement instructions; and a storage device adapted to store a plurality of instructions, the instructions adapted to be loaded and executed by the processor to: acquiring a target pedestrian based on a disparity map of a binocular camera; judging whether the target pedestrian is static: if yes, executing static operation, and judging whether forward collision early warning is performed or not; if not, executing non-static operation; the non-stationary operation comprises the steps of: respectively calculating a vehicle motion speed vector and a pedestrian motion speed vector based on the position change; judging whether the included angle between the motion velocity vector of the self vehicle and the motion velocity vector of the pedestrian is equal to 90 degrees or not: if yes, executing pedestrian crossing operation, and judging whether forward collision early warning is carried out or not; if not, executing the pedestrian non-crossing operation; the pedestrian non-crossing operation comprises the steps of: judging whether the included angle between the motion velocity vector of the self vehicle and the motion velocity vector of the pedestrian is equal to 0 degree or 180 degrees: if yes, executing the operation of the same-direction or opposite-direction movement of the self-vehicle and the pedestrian, and judging whether to carry out forward collision early warning; if not, the pedestrian obliquely-passing operation is executed, and whether forward collision early warning is carried out or not is judged.
The invention has the beneficial effects that: aiming at the condition that the obstacle is a pedestrian, a forward collision early warning overall strategy based on a binocular stereo camera is provided. The pedestrian that the vehicle marched the place ahead can effectively be discerned to it, under the people's different motion state of going, judges the possibility that the vehicle and its took place the collision and whether need carry out collision early warning immediately and remind the driver.
Drawings
FIG. 1 is a flow chart of a binocular stereo camera pedestrian collision warning algorithm;
fig. 2 is a schematic diagram of constructing a corresponding disparity map based on a binocular camera gray map;
FIG. 3 is a schematic view of a UV disparity map construction;
FIG. 4 is a diagram illustrating the correspondence between the original disparity map and the UV disparity map;
FIG. 5 is a schematic illustration of a pedestrian at rest;
FIG. 6 is a schematic illustration of a pedestrian crossing;
FIG. 7 is a schematic view of the co-directional movement of pedestrians;
FIG. 8 is a schematic view of the pedestrian's opposing motion;
fig. 9 is a schematic diagram of a pedestrian diagonal crossing movement.
Detailed Description
The invention is described in further detail below with reference to the figures and the examples, but without limiting the invention.
Whether the early warning is carried out or not is judged mainly according to the smaller value of the collision time TTC and the reinforced distance collision time ETTC and the time required by pedestrian crossing.
In the case of crossing (vertical crossing and inclined crossing), if the smaller value of TTC and ETTC is less than the crossing time and the smaller value is less than the threshold value, an alarm is given. That is, an alarm is given in the case where a collision occurs immediately before the pedestrian completes crossing and the collision is about to occur.
In the absence of crossing (stationary, co-directional or counter-directional travel), the alarm is given only if the smaller value of TTC, ETTC is less than the threshold. That is, an alarm is given in the event that a collision is imminent.
As shown in fig. 1, the embodiment of the pedestrian collision early warning method based on the binocular stereo camera comprises the following steps:
s1: acquiring an original image of a binocular camera, and calculating a disparity map;
according to the binocular camera imaging mathematical model, the original gray level image of the binocular camera is utilized, the parallax (only aiming at the overlapped and effective image area of the left camera and the right camera) is calculated point by point, and the parallax image corresponding to the original image is obtained. A large disparity indicates a close distance, and a small disparity indicates a far distance. And converting the disparity map into a spatial information point cloud map by using a mathematical model. As shown in fig. 2. The left side is the grey scale image of the left camera, the right side is the disparity map, and different grey values in the disparity map represent different disparities.
S2: detecting an obstacle based on the disparity map;
the calculation amount of detecting obstacles directly in the two-dimensional disparity map is large, and the two-dimensional disparity map can be converted into two one-dimensional disparity maps, namely a U disparity map and a V disparity map. Constructing a UV disparity map:
the number of lines of the U disparity map is equal to the number of columns of the V disparity map is equal to the maximum disparity value + 1;
The columns of the U disparity map correspond to the columns of the original disparity map;
the rows of the V disparity map correspond to the rows of the original disparity map;
in the V disparity map, (V, d) is the V-th line in the original disparity map, and the disparity is the number of d.
In the U-disparity map, (d, U) is the U-th column in the original disparity map, and the disparity is the number of d.
For example, the number of disparities 0,1,2,3,4, and 5 in the first line of the original disparity map is counted: 0, 2, 0,1, 1, 1, and so on. The schematic construction is shown in FIG. 3.
If we can find a straight line in the UV disparity map, the target corresponding to the straight line can be reversely found, and therefore the obstacle is detected. As shown in fig. 4.
Information (ID, location, size, etc.) about the obstacle is saved.
S3: and judging whether the barrier is in the alarm area. If yes, continuing the following steps;
and judging whether the alarm area exists according to the coordinates of the center position of the obstacle.
S4: judging whether the barrier is a pedestrian, if so, continuing the following steps;
and judging whether the rectangular frame of the barrier contains the pedestrian or not by utilizing a classifier trained in advance.
S5: acquiring a pedestrian with the closest distance;
if two or more pedestrian obstacles are obtained in step S4, the pedestrian closest to the own vehicle is identified.
S6: and judging whether the tracking frame number is larger than a preset threshold value. If not, returning to the step S1 and repeating the steps; if yes, continuing the following steps;
wherein a minimum threshold value of the number of tracking frames is set enough for the obtained data to be used for judging the traveling state of the pedestrian, such as whether the pedestrian is changed in position, stationary or moving, moving direction, moving speed, etc.
S7: judging whether the pedestrian is static: if yes, executing static operation; if not, executing non-static operation;
and comparing whether the coordinates of the central position of the pedestrian change between the current moment and the previous moment or not by using the coordinate system of the bicycle at the previous moment as a world coordinate system. No change, the pedestrian is stationary; and the pedestrian moves in a changed way. The pedestrian is stationary as shown in figure 5.
The stationary operation comprises the steps of:
s81: calculating TTC and ETTC, and selecting a smaller value t 1;
wherein the speed and the acceleration of the pedestrian are both 0.
S82: judging whether t1 is smaller than an alarm threshold value, if yes, carrying out forward collision early warning, judging that there is a collision risk, and reminding a driver to take necessary braking measures;
the non-stationary operation comprises the steps of:
s9: respectively calculating a vehicle motion velocity vector V1 and a pedestrian motion velocity vector V2 based on the position change;
Calculating a vehicle motion speed vector V1 according to the current vehicle center position coordinates (x2, y2) and the previous vehicle center position coordinates (x1, y 1); similarly, the pedestrian movement velocity vector V2 is calculated according to the pedestrian center position coordinates (x2, y2) at the current moment and the pedestrian center position coordinates (x1, y1) at the previous moment.
S10: judging whether the included angle between the vehicle motion velocity vector V1 and the pedestrian motion velocity vector V2 is equal to 90 degrees or not, if so, executing pedestrian crossing operation; if not, executing the pedestrian non-crossing operation;
wherein, an included angle between the vehicle motion velocity vector V1 and the pedestrian motion velocity vector V2 is calculated. If the included angle is equal to 90 degrees, it is determined that the pedestrian is crossing, as shown in fig. 6.
The pedestrian crossing operation includes the steps of:
s111: calculating TTC and ETTC, and selecting a smaller value t 2; calculating the pedestrian crossing time t 3;
wherein, the longitudinal relative speed is equal to the magnitude of the vehicle motion speed vector V1 and is used for calculating TTC and ETTC; the magnitude of the pedestrian movement velocity vector V2 is used to calculate the pedestrian crossing time t 3.
S112: judging whether t2 is smaller than t3, if not, not giving an alarm; if yes, continuing to execute the following steps;
s113: and (6) judging whether t2 is smaller than an alarm threshold value, and if so, alarming.
The pedestrian non-crossing operation comprises the steps of:
s12: it is determined whether the angle between the vehicle motion velocity vector V1 and the pedestrian motion velocity vector V2 is equal to 0 degree or 180 degrees. If yes, executing the operation of the same-direction or opposite-direction movement of the bicycle and the pedestrian; if not, the pedestrian obliquely-crossing operation is executed.
If the included angle between the vehicle motion velocity vector V1 and the pedestrian motion velocity vector V2 is equal to 0 degree, the vehicle and the pedestrian move in the same direction, as shown in fig. 7; equal to 180 degrees, the bicycle and the pedestrian move in opposite directions, as shown in fig. 8. If the angle θ between the vehicle moving velocity vector V1 and the pedestrian moving velocity vector V2 is not the specific angle mentioned above, the pedestrian is crossing diagonally as shown in fig. 9.
The operation of the same direction or opposite direction movement of the bicycle and the pedestrian comprises the following steps:
s131: calculating TTC and ETTC, and selecting a smaller value t 4;
wherein, the relative speed used for calculating TTC and ETTC is equal to | V1| - | V2|, namely the difference between the sizes of the vehicle motion speed vector V1 and the pedestrian motion speed vector V2 when moving in the same direction; equal to | V1| + | V2|, i.e., the sum of the magnitudes of the vehicle motion velocity vector V1 and the pedestrian motion velocity vector V2, in the opposite motion.
S132: judging whether t4 is smaller than an alarm threshold value, if not, not alarming; if yes, alarming.
The pedestrian oblique crossing comprises the following steps:
s141: calculating TTC and ETTC, and selecting a smaller value t 5; calculating the pedestrian crossing time t 6;
in specific calculation, the pedestrian movement velocity vector V2 is decomposed along the direction of the vehicle movement velocity vector V1 and the direction perpendicular to the vehicle movement velocity vector V1 to obtain component vectors V3 and V4, the magnitude of V3 is used for calculating TTC and ETTC, and the magnitude of V4 is used for calculating the pedestrian crossing time t 6.
S142: judging whether t5 is smaller than t6, if not, not giving an alarm; if yes, continuing to execute the following steps;
s143: and (6) judging whether t5 is smaller than an alarm threshold value, and if so, alarming.
Furthermore, it is within the scope of the present invention for a storage device having stored therein a plurality of instructions adapted to be loaded by a processor and to perform the above-described method. It is also within the scope of the present invention for the vehicle with the binocular stereo camera to have the memory device and the processor.
The above is the preferred embodiment of the present invention. It should be noted that, for a person skilled in the art, several modifications and refinements can be made without departing from the basic principle of the invention, and these modifications and refinements are also considered to be within the protective scope of the invention.
Claims (10)
1. A pedestrian collision early warning method based on a binocular stereo camera is characterized by comprising the following steps:
acquiring a target pedestrian based on a disparity map of a binocular camera;
judging whether the target pedestrian is static: if yes, executing static operation, and judging whether forward collision early warning is performed or not; if not, executing non-static operation;
the non-stationary operation comprises the steps of:
respectively calculating a vehicle motion velocity vector (V1) and a pedestrian motion velocity vector (V2) based on the position change;
judging whether the included angle between the vehicle motion velocity vector (V1) and the pedestrian motion velocity vector (V2) is equal to 90 degrees or not: if yes, executing pedestrian crossing operation, and judging whether forward collision early warning is carried out or not; if not, executing the pedestrian non-crossing operation;
the pedestrian non-crossing operation comprises the steps of:
judging whether the included angle between the vehicle motion velocity vector (V1) and the pedestrian motion velocity vector (V2) is equal to 0 degree or 180 degrees: if yes, executing the operation of the same-direction or opposite-direction movement of the self-vehicle and the pedestrian, and judging whether to carry out forward collision early warning; if not, the pedestrian obliquely-passing operation is executed, and whether forward collision early warning is carried out or not is judged.
2. The method of claim 1, wherein the quiescent operation comprises the steps of:
S81: calculating the Time To Collision (TTC) and the reinforcement distance time to collision (ETTC), and selecting the smaller value (t 1);
s82: and judging whether the smaller value (t1) is smaller than an alarm threshold value, if so, carrying out forward collision early warning, judging that the collision risk exists, and reminding a driver to take necessary braking measures.
3. The method of claim 1, wherein the pedestrian crossing operation comprises the steps of:
s111: calculating a Time To Collision (TTC) and an emphasized distance collision time (ETTC), selecting a smaller value (t2), calculating a pedestrian crossing time (t3), wherein a longitudinal relative velocity is equal to the magnitude of a vehicle moving velocity vector (V1) for calculating the Time To Collision (TTC) and the emphasized distance collision time (ETTC); the magnitude of the pedestrian movement velocity vector (V2) is used to calculate a pedestrian crossing time (t 3);
s112: judging whether the smaller value (t2) is smaller than the pedestrian crossing time (t3), if not, not giving an alarm; if yes, continuing to execute the following steps;
s113: and judging whether the smaller value (t2) is smaller than an alarm threshold value, and if so, alarming.
4. The method of claim 1, wherein the concurrent or opposite movement of the host vehicle and the pedestrian comprises the steps of:
s131: calculating a collision time (TTC) and an enhanced distance collision time (ETTC), and selecting a smaller value (t4), wherein an included angle between a vehicle motion speed vector (V1) and a pedestrian motion speed vector (V2) is equal to 0 degree, the vehicle motion speed vector is judged to move in the same direction and is equal to 180 degrees, the vehicle motion speed vector is judged to move in an opposite direction, and the relative speed between the vehicle and the pedestrian is equal to the difference between the vehicle motion speed vector (V1) and the pedestrian motion speed vector (V2) during the movement in the same direction; equal to the sum of the magnitudes of the vehicle motion velocity vector (V1) and the pedestrian motion velocity vector (V2) in the case of the opposite motion;
S132: judging whether the smaller value (t4) is smaller than an alarm threshold value, if not, not giving an alarm; if yes, alarming.
5. The method of claim 1, wherein the pedestrian cut-through operation comprises the steps of:
s141: calculating a Time To Collision (TTC) and an emphasized distance time to collision (ETTC), selecting a smaller value (t5), and calculating a pedestrian crossing time (t6), wherein a pedestrian movement velocity vector (V2) is resolved along a direction of a vehicle movement velocity vector (V1) and a direction perpendicular to the vehicle movement velocity vector (V1) to obtain a first component vector (V3) and a second component vector (V4), the magnitude of the first component vector (V3) is used for calculating the Time To Collision (TTC) and the emphasized distance time to collision (ETTC), and the magnitude of the second component vector (V4) is used for calculating the pedestrian crossing time (t 6);
s142: judging whether the smaller value (t5) is smaller than the crossing time (t6), if not, not giving an alarm; if yes, continuing to execute the following steps;
s143: and judging whether the smaller value (t5) is smaller than an alarm threshold value, and if so, alarming.
6. The method of claim 1, wherein the obtaining of the target pedestrian based on the disparity map of the binocular camera comprises the steps of:
s2: detecting an obstacle based on the disparity map;
S3: judging whether the barrier is in the alarm area, if so, continuing the following steps;
s4: judging whether the barrier is a pedestrian, if so, continuing the following steps;
s5: and acquiring the pedestrian with the closest distance as the target pedestrian.
7. The method according to claim 1, wherein the determining whether the target pedestrian is stationary is based on a coordinate system of the vehicle at the previous moment and a coordinate system of a central position of the target pedestrian is changed, no change, the pedestrian is stationary, the change is made, and the pedestrian moves by comparing the current moment with the previous moment; the calculated self-vehicle motion velocity vector (V1) and the pedestrian motion velocity vector (V2) are obtained by calculating the self-vehicle motion velocity vector (V1) according to the current self-vehicle central position coordinate and the last self-vehicle central position coordinate, and the pedestrian motion velocity vector (V2) is obtained by calculating according to the current target pedestrian central position coordinate and the last target pedestrian central position coordinate.
8. The method of claim 1, wherein before determining whether the target pedestrian is stationary, determining whether the tracking frame number is greater than a preset threshold, and if not, returning to the initial step; if yes, the following steps are continued.
9. A memory device having stored therein a plurality of instructions adapted to be loaded and executed by a processor to:
acquiring a target pedestrian based on a disparity map of a binocular camera;
judging whether the target pedestrian is static: if yes, executing static operation, and judging whether forward collision early warning is performed or not; if not, executing non-static operation;
the non-stationary operation comprises the steps of:
respectively calculating a vehicle motion velocity vector (V1) and a pedestrian motion velocity vector (V2) based on the position change;
judging whether the included angle between the vehicle motion velocity vector (V1) and the pedestrian motion velocity vector (V2) is equal to 90 degrees or not: if yes, executing pedestrian crossing operation, and judging whether forward collision early warning is carried out or not; if not, executing the pedestrian non-crossing operation;
the pedestrian non-crossing operation comprises the steps of:
judging whether the included angle between the vehicle motion velocity vector (V1) and the pedestrian motion velocity vector (V2) is equal to 0 degree or 180 degrees: if yes, executing the operation of the same-direction or opposite-direction movement of the self-vehicle and the pedestrian, and judging whether to carry out forward collision early warning; if not, the pedestrian obliquely-passing operation is executed, and whether forward collision early warning is carried out or not is judged.
10. An automobile with a binocular stereo camera is characterized by comprising:
A processor adapted to implement instructions; and
a storage device adapted to store a plurality of instructions, the instructions adapted to be loaded and executed by a processor to:
acquiring a target pedestrian based on a disparity map of a binocular camera;
judging whether the target pedestrian is static: if yes, executing static operation, and judging whether forward collision early warning is performed or not; if not, executing non-static operation;
the non-stationary operation comprises the steps of:
respectively calculating a vehicle motion velocity vector (V1) and a pedestrian motion velocity vector (V2) based on the position change;
judging whether the included angle between the vehicle motion velocity vector (V1) and the pedestrian motion velocity vector (V2) is equal to 90 degrees or not: if yes, executing pedestrian crossing operation, and judging whether forward collision early warning is carried out or not; if not, executing the pedestrian non-crossing operation;
the pedestrian non-crossing operation comprises the steps of:
judging whether the included angle between the vehicle motion velocity vector (V1) and the pedestrian motion velocity vector (V2) is equal to 0 degree or 180 degrees: if yes, executing the operation of the same-direction or opposite-direction movement of the self-vehicle and the pedestrian, and judging whether to carry out forward collision early warning; if not, the pedestrian obliquely-passing operation is executed, and whether forward collision early warning is carried out or not is judged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010497044.5A CN111845554A (en) | 2020-06-03 | 2020-06-03 | Pedestrian collision early warning method and device based on binocular stereo camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010497044.5A CN111845554A (en) | 2020-06-03 | 2020-06-03 | Pedestrian collision early warning method and device based on binocular stereo camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111845554A true CN111845554A (en) | 2020-10-30 |
Family
ID=72985449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010497044.5A Pending CN111845554A (en) | 2020-06-03 | 2020-06-03 | Pedestrian collision early warning method and device based on binocular stereo camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111845554A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113787961A (en) * | 2021-08-11 | 2021-12-14 | 普行智能停车（深圳）有限公司 | Alarm method, device, equipment and medium for sudden obstacle appearing during vehicle running |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102008062916A1 (en) * | 2008-12-23 | 2010-06-24 | Continental Safety Engineering International Gmbh | Method for determining a collision probability of a vehicle with a living being |
| DE102009035072A1 (en) * | 2009-07-28 | 2011-02-10 | Bayerische Motoren Werke Aktiengesellschaft | Motor vehicle collision warning system has a transceiver working with a transponder at the object in the path, e.g. a pedestrian, to determine its position and predict its future position |
| CN108216226A (en) * | 2016-12-22 | 2018-06-29 | 丰田自动车株式会社 | Collision elimination auxiliary device |
| CN108263279A (en) * | 2016-12-30 | 2018-07-10 | 现代自动车株式会社 | The pedestrian detection and pedestrian impact avoiding device and method integrated based on sensor |
| WO2019179464A1 (en) * | 2018-03-23 | 2019-09-26 | 北京市商汤科技开发有限公司 | Method for predicting direction of movement of target object, vehicle control method, and device |
| CN110481544A (en) * | 2019-07-30 | 2019-11-22 | 江苏大学 | A kind of automotive correlation prevention method and anti-collision system for pedestrian |
| CN110816523A (en) * | 2018-08-10 | 2020-02-21 | 现代自动车株式会社 | Vehicle and control method thereof |
| CN113753070A (en) * | 2020-06-04 | 2021-12-07 | 百度（美国）有限责任公司 | Locked pedestrian detection and prediction for automated vehicles |
-
2020
- 2020-06-03 CN CN202010497044.5A patent/CN111845554A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102008062916A1 (en) * | 2008-12-23 | 2010-06-24 | Continental Safety Engineering International Gmbh | Method for determining a collision probability of a vehicle with a living being |
| DE102009035072A1 (en) * | 2009-07-28 | 2011-02-10 | Bayerische Motoren Werke Aktiengesellschaft | Motor vehicle collision warning system has a transceiver working with a transponder at the object in the path, e.g. a pedestrian, to determine its position and predict its future position |
| CN108216226A (en) * | 2016-12-22 | 2018-06-29 | 丰田自动车株式会社 | Collision elimination auxiliary device |
| CN108263279A (en) * | 2016-12-30 | 2018-07-10 | 现代自动车株式会社 | The pedestrian detection and pedestrian impact avoiding device and method integrated based on sensor |
| WO2019179464A1 (en) * | 2018-03-23 | 2019-09-26 | 北京市商汤科技开发有限公司 | Method for predicting direction of movement of target object, vehicle control method, and device |
| CN110816523A (en) * | 2018-08-10 | 2020-02-21 | 现代自动车株式会社 | Vehicle and control method thereof |
| CN110481544A (en) * | 2019-07-30 | 2019-11-22 | 江苏大学 | A kind of automotive correlation prevention method and anti-collision system for pedestrian |
| CN113753070A (en) * | 2020-06-04 | 2021-12-07 | 百度（美国）有限责任公司 | Locked pedestrian detection and prediction for automated vehicles |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113787961A (en) * | 2021-08-11 | 2021-12-14 | 普行智能停车（深圳）有限公司 | Alarm method, device, equipment and medium for sudden obstacle appearing during vehicle running |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ziebinski et al. | Review of advanced driver assistance systems (ADAS) | |
| US9121717B1 (en) | Collision avoidance for vehicle control | |
| US9884623B2 (en) | Method for image-based vehicle localization | |
| CN108496178B (en) | System and method for estimating future path | |
| US10246030B2 (en) | Object detection apparatus and driving assistance apparatus | |
| US9568611B2 (en) | Detecting objects obstructing a driver's view of a road | |
| US9058247B2 (en) | Risk potential calculation apparatus | |
| US20170206440A1 (en) | Fixation generation for machine learning | |
| JP2021500652A (en) | Pedestrian behavior prediction for autonomous vehicles | |
| Guo et al. | A multimodal ADAS system for unmarked urban scenarios based on road context understanding | |
| Labayrade et al. | In-vehicle obstacles detection and characterization by stereovision | |
| JP6073358B2 (en) | Active warning and / or navigation assistance method for avoiding collision between vehicle body and / or wheel and object | |
| JP2009037622A (en) | Method and device for evaluating image | |
| CN104509099B (en) | Three-dimensional body detection device and three-dimensional body detection method | |
| CN114258366A (en) | Polyline profile representation for autonomous vehicles | |
| CN113844445B (en) | Automatic emergency braking system and method for vehicle based on prediction reference line coordinate system | |
| CN112758067A (en) | Emergency braking method and system based on binocular stereo camera and intelligent terminal | |
| CN111497741A (en) | Collision early warning method and device | |
| JP7130580B2 (en) | Road surface detector | |
| Altay et al. | Lidar data analysis for time to headway determination in the drivesafe project field tests | |
| CN111845554A (en) | Pedestrian collision early warning method and device based on binocular stereo camera | |
| CN114022866A (en) | Method and device for estimating pose and speed of obstacle | |
| CN113838060A (en) | Perception system for autonomous vehicle | |
| CN111191538B (en) | Obstacle tracking method, device and system based on binocular camera and storage medium | |
| CN116524454A (en) | Object tracking device, object tracking method, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |