CN114371707B - Pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction - Google Patents

Pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction Download PDF

Info

Publication number
CN114371707B
CN114371707B CN202111669198.9A CN202111669198A CN114371707B CN 114371707 B CN114371707 B CN 114371707B CN 202111669198 A CN202111669198 A CN 202111669198A CN 114371707 B CN114371707 B CN 114371707B
Authority
CN
China
Prior art keywords
pedestrian
vehicle
collision avoidance
force
road
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111669198.9A
Other languages
Chinese (zh)
Other versions
CN114371707A (en
Inventor
唐斌
杨铮奕
胡子添
江浩斌
蔡英凤
袁朝春
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Kingyoung Intelligent Science And Technology Co ltd
Original Assignee
Nanjing Kingyoung Intelligent Science And Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Kingyoung Intelligent Science And Technology Co ltd filed Critical Nanjing Kingyoung Intelligent Science And Technology Co ltd
Priority to CN202111669198.9A priority Critical patent/CN114371707B/en
Publication of CN114371707A publication Critical patent/CN114371707A/en
Application granted granted Critical
Publication of CN114371707B publication Critical patent/CN114371707B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0223—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving speed control of the vehicle
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Electromagnetism (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention discloses a pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction, which are used for identifying the face orientation of a pedestrian in a front image of a vehicle to judge whether the pedestrian notices an incoming vehicle or not, and fusing pedestrian motion state information, the face orientation of the pedestrian and vehicle motion state information to judge the intention of the pedestrian; predicting pedestrian motion of an unobserved vehicle by using a Markov pedestrian model; simultaneously introducing a social force model and a Markov pedestrian model to predict the pedestrian motion which is noticed by the vehicle and is continuously walking, and carrying out weighted fusion on the predicted results of the two models to obtain corrected pedestrian positions, so as to obtain a track curve in preset duration; judging the safety state of the track of the pedestrian, and deciding a corresponding collision avoidance strategy; meanwhile, a pedestrian track prediction and active collision avoidance system considering human-vehicle interaction is constructed; the method and the system designed by the invention can improve the safety and the stability of the intelligent automobile, so that the whole longitudinal and transverse collision avoidance decision system is more perfect and effective.

Description

Pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction
Technical Field
The invention belongs to the technical field of intelligent driving safety, and particularly relates to a pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction.
Background
With the continuous progress and development of the global industry, the automobile conservation amount is in a continuous increasing trend, so that the road traffic situation is more severe. In urban road scenes, pedestrians which are easy to be injured are taken as individuals moving independently, and when crossing a road, collision or collision is easy to happen with vehicles which come and go on the road, so that traffic accidents are frequent.
The existing research on the pedestrian collision avoidance method can be roughly divided into four aspects of pedestrian detection and identification, prediction, risk assessment, collision avoidance and the like. In the aspect of pedestrian track prediction, the main problems of the existing method are that complex and changeable motion characteristics of pedestrians cannot be considered, influence of surrounding environments on the motion state of the pedestrians is ignored, and the predicted pedestrian track is greatly different from the real pedestrian track. In the aspect of collision avoidance methods, most of the existing methods adopt independent longitudinal collision avoidance or transverse collision avoidance, the two modes have respective adaptive working conditions, and the variable behaviors of pedestrians can cause interference to collision avoidance decisions of intelligent automobiles.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a pedestrian track prediction and collision avoidance control method and system considering human-vehicle interaction; the method can reasonably predict the future motion trail of the pedestrian and make collision avoidance decision for dangerous states, can ensure stable and safe running of the vehicle, can protect the pedestrian and reduce road traffic accidents.
In order to solve the problems, the invention adopts the following technical scheme,
The pedestrian track prediction and vehicle active collision avoidance method considering human-vehicle interaction comprises the following steps:
s1, acquiring the running state information of the vehicle, the pedestrian movement state information and a front image of the vehicle.
S2, extracting a pedestrian head area and identifying the face direction of the pedestrian according to the acquired front image of the vehicle, and judging whether the pedestrian notices an incoming vehicle or not according to the face direction of the pedestrian; based on the logistic regression model, pedestrian motion state information, pedestrian face orientation and vehicle motion state information are fused, and the intention of the pedestrian is judged.
S3, according to the face orientation of the pedestrian, if the pedestrian is judged to not notice the vehicle, predicting the track of the pedestrian in the future preset duration by using a Markov pedestrian model;
S4, according to the face orientation of the pedestrian and the pedestrian intention judgment result, if the pedestrian is judged to notice the vehicle and continue to walk, a social force model is introduced to predict the motion of the pedestrian at the moment, and a Markov pedestrian model is introduced to predict the motion process of the pedestrian; the pedestrian position predicted by the Markov pedestrian model and the pedestrian position predicted by the social force model are subjected to weighted fusion to obtain a corrected pedestrian position; and obtaining a track curve in the preset duration according to the set time step.
S5, judging the safety state according to the predicted pedestrian track, and deciding a proper collision avoidance strategy of the vehicle under the condition of ensuring safety.
Further, in the above S2, the step of identifying the face orientation of the pedestrian includes:
(1) And detecting the head area of the pedestrian by adopting Yolo algorithm according to the front image of the vehicle to obtain a head image.
(2) And constructing a forward/lateral/back classifier of the pedestrian face by adopting a convolutional neural network, and determining the Orientation of the pedestrian face.
Further, in the above S2, the logistic regression model is used for pedestrian intention judgment, and includes the following steps:
(1) The pedestrian speed v ped, the pedestrian face Orientation degree, the vehicle speed v vehicle and the human-vehicle Distance are taken as independent variables, the intention of the pedestrian to select to walk or stop when facing an incoming vehicle in the human-vehicle interaction process is taken as the dependent variable, and the function expression in the logistic regression model for judging the pedestrian intention is as follows:
Wherein, the independent variable x= [ c, v ped,Orientation,vvehicle,Distance]T, c is any constant term. θ= [ θ 0,θ1,θ2,θ3,θ4]T ] is a coefficient set, θ i is a coefficient corresponding to the i-th argument, i=0, 1,2,3, 4; obtaining the coefficient corresponding to the ith independent variable as a minimum value theta i of the cost function through a gradient descent method; the expression formula of the cost function is as follows:
Where J (θ) is the cost function, m is the number of samples, H θ is the hypothesized value, y (i) is the ith real value, x (i) is the ith argument, x (i) ε x.
(2) And judging the intention of the pedestrian by using the trained logistic regression model, wherein if H θ is more than 0.5, H θ is less than 0.5, the pedestrian stops.
Further, in the above step S3, the process of predicting the track of the pedestrian within the preset duration in the future by using the markov pedestrian model is as follows:
The pedestrian does not notice the vehicle, at this time, the vehicle on the road can be regarded as no interference to the pedestrian, the free motion of the pedestrian passing through the street under the condition of no external interference accords with the Markov process, the future position and speed of the pedestrian depend on the current position and speed of the pedestrian, and the state description of the pedestrian can be obtained:
Stateped＝(xp(t),yp(t),vx-p(t),vy-p(t))
Where State ped is the State of the pedestrian, v x-p (t) and v y-p (t) represent the speeds of the pedestrian in the X-direction and Y-direction at time t, respectively, deltav x-p and Deltav y-p represent the speed increment in the X-direction and Y-direction at time t, v x-p (t+Deltat) and v y-p (t+Deltat) represent the speeds of the pedestrian in the X-direction and Y-direction at+Deltat, respectively, k x and k y represent constants, AndRepresenting the expected speed of the pedestrian in the X direction and the Y direction respectively, epsilon x and epsilon y representing the random disturbance of the speed increment of the pedestrian in the X direction and the Y direction respectively, X p (t) and Y p (t) representing the displacement function of the pedestrian in the X direction and the Y direction relative to time t respectively, and p m (t) representing the position of the pedestrian at t predicted by the Markov pedestrian model.
Further, in the above step S4, the step of predicting the motion of the pedestrian by using the social force model includes:
The pedestrian moves to the target point and has a driving force, the pedestrian can be subjected to the repulsive force of the vehicle facing the incoming vehicle, the road can also apply a hidden boundary force to the pedestrian, the forces are accumulated to form a resultant force, and the position of the pedestrian is recursively calculated along with the time step under the action of the social force. The expression formula of the resultant force and the position of the pedestrian is as follows:
Where F ped (t) represents the resultant social force applied to the pedestrian, F d represents the driving force applied to the pedestrian to move toward the target point, F vp represents the repulsive force applied to the pedestrian facing the incoming vehicle, and F e represents the road to apply a hidden boundary force to the pedestrian. v n (t) represents the speed of the pedestrian at t, p s (t) represents the position of the pedestrian at t predicted by the social force model, p s (t+Δt) represents the position of the pedestrian at t+Δt, Δt represents the time step, and m represents the mass of the pedestrian.
Further, in the above step S4, the movement of the pedestrian crossing at this time is considered as a combination of free non-interfering movement and interfering movement of the coming vehicle, and the predicted pedestrian position is obtained by weighting the pedestrian position predicted by the markov pedestrian model and the pedestrian position predicted by the social force model, which is expressed by the following formula:
p(t)＝τm·pm(t)+τs·ps(t)
Where p (t) represents the pedestrian position at t obtained by fusing and correcting the markov pedestrian model and the social force model, τ m and τ s represent the weight coefficients of p m (t) and p s (t), respectively.
Further, in the step S5, the method for judging the safety state of the pedestrian and the vehicle comprises:
(1) Dividing the position of pedestrians on a road into a dangerous area, a high-risk area and a safe area;
(2) Calculating the longitudinal collision avoidance time of the vehicle, and considering the position fluctuation of the pedestrian position in the longitudinal direction, wherein the obtained longitudinal collision avoidance time of the vehicle is a time region range:
Wherein t v represents the longitudinal collision avoidance time of the vehicle, v vehicle represents the vehicle speed, Δs represents the longitudinal relative distance between the vehicle and the pedestrian, ε represents the longitudinal offset when the pedestrian crosses the street, and κ represents the time elasticity factor;
(3) Uniformly sampling the obtained time zone segment to obtain a series of time sequence points:
{t-κ,t-κ+Δt’,t-κ+2Δt’,…,t-κ+(n-1)Δt’,t+κ}
wherein Δt' is the time interval of uniform sampling;
(4) Substituting the time sequence points into a position expression of the predicted track to generate a position sequence in the time region:
{Pt-κ,Pt-κ+Δt',Pt-κ+2Δt',…,Pt+κ-Δt',Pt+κ}
(5) And deciding a collision avoidance strategy of the vehicle according to the number of the position sequence points in the three areas obtained by dividing.
Further, the collision avoidance strategy of the vehicle is as follows:
1) If all the sequence points fall in the dangerous area, performing transverse collision avoidance operation;
2) If all the sequence points fall in the high risk area, performing longitudinal collision avoidance operation;
3) As long as one sequence point falls in the high risk area, a longitudinal collision avoidance operation is performed.
If the transverse collision avoidance operation is determined, the transverse planning layer plans a transverse collision avoidance path according to the state of the pedestrians relative to the vehicle; if the longitudinal collision avoidance operation is determined, an emergency braking or deceleration method is adopted to achieve the purpose of the longitudinal collision avoidance operation.
Further, the specific steps of the transverse planning layer for planning the transverse collision avoidance path are as follows:
(1) Combining the pedestrian track predicted by the S3 or the S4, and dividing a transverse collision avoidance path by adopting an artificial potential field rule; constructing an attractive potential field, a road boundary repulsive potential field and an elliptical obstacle repulsive potential field:
(2) And (3) carrying out negative gradient on the attraction potential field, the road boundary repulsive potential field and the elliptical obstacle repulsive potential field to obtain potential field force corresponding to each potential field, wherein the potential field force is as follows:
adding the three potential field forces to obtain the resultant force born by the vehicle:
Ftotal＝Falt+Froad+Fobs
Wherein U alt represents a gravitational potential field to which the vehicle is subjected, U road represents a road boundary repulsive potential field to which the vehicle is subjected, U obs represents an obstacle repulsive potential field to which the vehicle is subjected, F alt represents gravitational force, F road represents a road boundary repulsive force, F obs represents an obstacle repulsive force, K alt represents a gravitational potential field gain coefficient, X represents a vehicle real-time coordinate, X g represents a vehicle target point coordinate, K road represents a road boundary constraint coefficient, X represents a vehicle coordinate in an X-direction, y represents a vehicle coordinate in a y-axis direction, y road,i represents an ordinate of an i-th road boundary line, W represents a vehicle width, (X obs,yobs) represents an obstacle coordinate, sigma x and sigma y represent distance influencing factors by which the obstacle acts on the vehicle, For representing gradient calculations;
(3) The position points where the vehicle moves under the action of the force of the combined force can be obtained by the force balance of the potential field force in the transverse direction, and the transverse planning path of obstacle avoidance can be obtained by curve fitting of the points;
(4) According to the position change in the process of crossing the pedestrian, carrying out collision risk analysis on the vehicle and the pedestrian in real time, and judging whether a path is required to be re-planned or not, so as to obtain a real-time transverse collision avoidance path; ; the collision risk analysis is performed by the vehicle and the pedestrian according to whether the position area where the pedestrian may appear and the position area where the vehicle may appear overlap in the planning period, if so, the collision risk is considered to exist, and the path is required to be re-planned, otherwise, the collision risk is considered not to exist, and the path is not required to be re-planned.
Further, the specific steps of the emergency braking or decelerating method are as follows:
(1) If emergency braking is adopted, a fuzzy controller is established, the relative speed and the relative distance between the vehicle and the pedestrian are used as input, and the expected deceleration is output;
(2) If the pedestrian is decelerated to avoid and normally runs after passing through the road, the expected minimum deceleration of the vehicle is obtained by the following formula:
Wherein:
Where t pass denotes a time for a pedestrian to traverse a road, L path denotes a length of the road, Y p denotes coordinates of the pedestrian in the Y direction, v p-y denotes a speed of the pedestrian in the Y direction, v vehicle denotes a speed of the vehicle, v ped denotes a speed of the pedestrian, Δs denotes a longitudinal relative distance between the pedestrian and the vehicle, and a denotes a desired minimum deceleration.
A pedestrian track prediction and collision avoidance control system considering human-vehicle interaction specifically comprises an environment sensing module, a pedestrian intention judging module, a pedestrian track prediction module and a longitudinal collision avoidance decision module.
The environment sensing module is used for acquiring the running state information of the vehicle, the moving state information of the pedestrians and the front image of the vehicle, and is respectively connected with the pedestrian intention judging module, the pedestrian track predicting module and the longitudinal and transverse collision avoidance decision module;
The pedestrian intention judging module detects the head of a pedestrian in the front image of the vehicle by utilizing the image processing unit according to the running state information, the pedestrian movement state information and the front image of the vehicle, which are acquired by the environment sensing module, so as to recognize the face orientation of the pedestrian crossing the street; judging the intention of the pedestrian crossing based on the recognition result of the face orientation of the pedestrian, and sending the intention judgment result to a pedestrian track prediction module;
The pedestrian track prediction module receives the running state information of the vehicle, the moving state information of the pedestrians and the judging result of the pedestrian intention judging module, which are sent by the environment sensing module, and predicts the track of the pedestrians in the future preset time length by using a Markov pedestrian model for the pedestrians which do not notice the vehicle; regarding pedestrians who notice vehicles and continue to walk, regarding the movement of pedestrians crossing the street as a combination of free movement and interference movement of coming vehicles; introducing a social force model into the motion prediction of the pedestrians at the moment, and carrying out weighted fusion on the pedestrian position predicted by the prior Markov pedestrian model and the pedestrian position predicted by the social force model to obtain a corrected pedestrian position; and obtaining a track curve in a preset time length according to the set time step length, and sending the predicted track curve to a longitudinal and transverse collision avoidance decision module.
The longitudinal and transverse collision avoidance decision module receives the pedestrian track predicted by the pedestrian track prediction module, evaluates the feasibility of longitudinal collision avoidance and transverse collision avoidance through the safety state analysis of the pedestrians and the vehicles, and decides a proper collision avoidance strategy of the vehicles under the condition of ensuring the safety.
Further, the environment sensing module comprises a GPS, a speed sensor, a laser radar and a monocular camera which are arranged on the vehicle, and position information and speed information of pedestrians crossing the street relative to the vehicle are obtained in real time; the monocular camera acquires an image in front of the vehicle.
The invention has the beneficial effects that:
1. The method adopts a mode of combining a Markov pedestrian model and a social force model, considers the combination of free motion and interference motion of the coming vehicles when pedestrians cross the street, predicts the motion trail of the pedestrians in the environment without external interference and the environment with human-vehicle interaction interference, improves the accuracy of trail prediction, and enables the prediction result to be more close to the real motion trail of the pedestrians.
2. The longitudinal and transverse collision avoidance selection strategy designed according to the predicted pedestrian track can adapt to pedestrian collision avoidance requirements with behavior randomness, and ensure that the proper collision avoidance strategy is selected in advance under the safety of the vehicle, so that the occurrence of collision accidents of the vehicles and the people is reduced, and the road safety is effectively improved.
Drawings
FIG. 1 is a block diagram of a pedestrian trajectory prediction and vehicle active collision avoidance system that takes human-vehicle interaction into account in accordance with an embodiment of the present invention;
FIG. 2 is a flow chart of a method for pedestrian trajectory prediction and vehicle active collision avoidance that takes into account human-vehicle interactions according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart for identifying pedestrian face orientation;
FIG. 4 is a flow chart for pedestrian intent determination;
Fig. 5 is a division diagram of a pedestrian risk area.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
For an analysis object, a straight line which is parallel to the right road direction and is taken as an X axis, a straight line which is perpendicular to the road direction and is taken as a Y axis, and a vehicle centroid point is taken as a coordinate system origin. The positions of the vehicle and the pedestrian are represented by coordinates in a coordinate system.
As shown in fig. 2, consider a flow chart of a method for predicting a pedestrian trajectory and actively avoiding collision of a vehicle by human-vehicle interaction. The specific flow is as follows:
s1, acquiring the running state information of the vehicle, the pedestrian movement state information and a front image of the vehicle.
And S2, extracting a pedestrian head area by utilizing Yolo algorithm according to the acquired front image of the vehicle, building a convolutional neural network to identify the face direction of the pedestrian, and judging whether the pedestrian notices an incoming vehicle according to the face direction of the pedestrian. Based on the logistic regression model, pedestrian motion state information, pedestrian face orientation and vehicle motion state information are fused, and the intention of the pedestrian is judged and used as part of S4 input.
And S3, according to the face orientation of the pedestrian, if the pedestrian is judged to not notice the vehicle, predicting the track of the pedestrian in the future preset duration by using a Markov pedestrian model.
S4, according to the face orientation of the pedestrian and the pedestrian intention judgment result, if the pedestrian is judged to notice the vehicle and continue to walk, the motion of the pedestrian crossing the street is considered as the combination of free non-interference motion and interference motion of the coming vehicle, the motion prediction of the pedestrian at the moment is introduced into a social force model, and a Markov pedestrian model is introduced into the motion process of the pedestrian to predict; and carrying out weighted fusion on the pedestrian position predicted by the Markov pedestrian model and the pedestrian position predicted by the social force model to obtain the corrected pedestrian position. And obtaining a track curve in the preset duration according to the set time step.
S5, judging the safety state according to the predicted pedestrian track, and deciding a proper collision avoidance strategy of the vehicle under the condition of ensuring safety.
Further, in the above S3, the specific process of predicting the track of the pedestrian within the preset time period in the future by using the markov pedestrian model includes:
The pedestrian does not notice the vehicle, at this time, the vehicle on the road can be regarded as no interference to the pedestrian, the free motion of the pedestrian passing through the street under the condition of no external interference accords with the Markov process, the future position and speed of the pedestrian depend on the current position and speed of the pedestrian, and the state description of the pedestrian can be obtained:
Stateped＝(xp(t),yp(t),vx-p(t),vy-p(t))
Where State ped is the State of the pedestrian, v x-p (t) and v y-p (t) represent the speeds of the pedestrian in the X-direction and Y-direction at time t, respectively, deltav x-p and Deltav y-p represent the speed increment in the X-direction and Y-direction at time t, v x-p (t+Deltat) and v y-p (t+Deltat) represent the speeds of the pedestrian in the X-direction and Y-direction at+Deltat, respectively, k x and k y represent constants, AndRepresenting the expected speed of the pedestrian in the X direction and the Y direction respectively, epsilon x and epsilon y representing the random disturbance of the speed increment of the pedestrian in the X direction and the Y direction respectively, X p (t) and Y p (t) representing the displacement function of the pedestrian in the X direction and the Y direction relative to time t respectively, and p m (t) representing the position of the pedestrian at t predicted by the Markov pedestrian model.
Further, in the above S4, the pedestrian notices the vehicle and intends to walk in the face of the coming vehicle, and the motion prediction of the pedestrian introduces a social force model at this time, and the specific steps include:
The pedestrian moves to the target point and has a driving force, the pedestrian can be subjected to the repulsive force of the vehicle facing the incoming vehicle, the road can also apply a hidden boundary force to the pedestrian, the forces are accumulated to form a resultant force, and the position of the pedestrian is recursively calculated along with the time step under the action of the social force. The expression formula of the resultant force and the position of the pedestrian is as follows:
Where F ped (t) represents the resultant social force applied to the pedestrian, F d represents the driving force applied to the pedestrian to move toward the target point, F vp represents the repulsive force applied to the pedestrian facing the incoming vehicle, and F e represents the road to apply a hidden boundary force to the pedestrian. v n (t) represents the speed of the pedestrian at t, p s (t) represents the position of the pedestrian at t predicted by the social force model, p s (t+Δt) represents the position of the pedestrian at t+Δt, Δt represents the time step, and m represents the mass of the pedestrian.
Further, in the above step S4, the movement of the pedestrian crossing at this time is considered as a combination of free non-interfering movement and interfering movement of the coming vehicle, and the predicted pedestrian position is obtained by weighting the pedestrian position predicted by the markov pedestrian model and the pedestrian position predicted by the social force model, which is expressed by the following formula:
p(t)＝τm·pm(t)+τs·ps(t)
Where p (t) represents the pedestrian position at t obtained by fusing and correcting the markov pedestrian model and the social force model, τ m and τ s represent the weight coefficients of p m (t) and p s (t), respectively.
In the process of crossing the street, pedestrians are not noticed to be in a non-interference state when the vehicle is crossed, the movement direction of the pedestrians is changed less, and the predicted track has a certain error due to the fact that the social force model considers the external environment, so that the result of predicting the Markov pedestrian model is better than the social force model, and the Markov pedestrian model is directly adopted to model the movement of the pedestrians at the moment. The pedestrian notices that the movement state of the incoming vehicle changes, the influence of the vehicle on the pedestrian is considered by using the social force model, and the pedestrian position predicted by the Markov pedestrian model and the pedestrian position predicted by the social force model are added to obtain the corrected pedestrian position. And respectively obtaining a predicted track curve of the pedestrian which does not notice the vehicle and a predicted track curve of the pedestrian which notices the vehicle and continues to walk in the preset time length according to the set time step.
As shown in fig. 3, a flow chart for recognizing face orientation is shown. And using the front image of the vehicle acquired by the monocular camera as an input image, and detecting the head region of the pedestrian by using Yolo algorithm. And constructing a pedestrian face orientation classifier by using a convolutional neural network, and identifying a face orientation state of the detected pedestrian head area. The method comprises the following steps:
(1) After the pedestrian image is acquired by the monocular camera, a Yolo network structure is set for adjusting training parameters, and a Yolo algorithm is adopted to detect the head area of the pedestrian, so that a head image is obtained.
(2) In order to detect whether a pedestrian notices an incoming car, it is necessary to recognize the face orientation of the pedestrian. A forward/lateral/backward classifier of the face of the pedestrian is built by adopting a convolutional neural network, and the classifier adopts four convolutional layers to carry out convolution operation with 2 multiplied by 2,3 multiplied by 3, 5 multiplied by 5 and 7 multiplied by 7 convolution kernels. Normalization and activation operations are performed using batch normalization (Batch Normalization) and a ReLU activation function. Redundant information in the feature map obtained by convolution is relieved through pooling operation of a maximum pooling layer with the step length of 2 by 3×3. And replacing the full-connection layer with a2 multiplied by 2 convolution layer, classifying the output by adopting a Softmax function, calculating the probability of each direction, and finally determining the face direction of the pedestrian. The probability expression formula for calculating the class j of the image is as follows:
where ζ represents a parameter of the network structure, and f i represents a feature of learning the i-th image.
As shown in fig. 4, a flow chart for pedestrian intention judgment is shown. Four factors of pedestrian speed, pedestrian face orientation, automobile speed and distance between pedestrians and vehicles are taken as independent variables, whether pedestrians stop along with incoming vehicles in the human-vehicle interaction process are selected as the independent variables, and behaviors of pedestrians crossing the street are identified by utilizing a logistic regression model, and specifically:
(1) The pedestrian speed v ped, the pedestrian face Orientation degree, the vehicle speed v vehicle and the human-vehicle Distance are taken as independent variables, the intention of the pedestrian to select to walk or stop when facing an incoming vehicle in the human-vehicle interaction process is taken as the dependent variable, and the function expression in the logistic regression model for judging the pedestrian intention is as follows:
wherein, the independent variable x= [ c, v ped,Orientation,vvehicle,Distance]T, c is any constant term. θ= [ θ 0,θ1,θ2,θ3,θ4]T ] is a coefficient set, and θ i is coefficients i=0, 1,2, 3,4 corresponding to the i-th argument. In this embodiment, the coefficient set θ is obtained by obtaining the minimum value of the cost function by the gradient descent method. The expression formula of the cost function is as follows:
Where m is the number of samples, H θ is the hypothetical value, and y is the actual value.
(2) The trained logistic regression model is used for judging the intention of the pedestrian, H θ is more than 0.5, H θ is less than 0.5, and the pedestrian stops.
FIG. 5 is a division of pedestrian hazard areas, with the area from the right edge of the vehicle to the right edge of the road being the hazard area; the area directly in front of the vehicle may be regarded as a high risk area; other areas on the road may be considered safe areas. And determining that the vehicle selects a proper collision avoidance strategy under the condition of ensuring the safety by judging the position of the area where the pedestrian is on the road and the safety state between the pedestrian and the vehicle. The designed longitudinal and transverse collision avoidance selection strategy comprises the following steps:
(1) The pedestrian's position on the road is divided into a dangerous area, a high risk area, and a safe area.
(2) And calculating the time for the longitudinal collision avoidance of the vehicle. Considering the position fluctuation of the pedestrian position in the longitudinal direction, the obtained longitudinal collision avoidance time of the vehicle is a time zone range:
Where t v denotes a vehicle longitudinal collision avoidance time, v vehicle denotes a vehicle speed, Δs denotes a longitudinal relative distance between the vehicle and the pedestrian, ε denotes a longitudinal offset when the pedestrian crosses the street, and κ denotes a time elastic factor.
(3) Uniformly sampling the obtained time zone segment to obtain a series of time sequence points:
{t-κ,t-κ+Δt’,t-κ+2Δt’,…,t-κ+(n-1)Δt’,t+κ}
Where Δt' is the time interval of uniform sampling.
(4) Substituting the time sequence points into a position expression of the predicted track to generate a position sequence in the time region:
{Pt-κ,Pt-κ+Δt',Pt-κ+2Δt',…,Pt+κ-Δt',Pt+κ}
(5) And deciding a collision avoidance strategy of the vehicle according to the number of the position sequence points in the three areas obtained by dividing.
1) If all the sequence points fall in the dangerous area, performing transverse collision avoidance operation;
2) If all the sequence points fall in the high risk area, performing longitudinal collision avoidance operation;
3) As long as one sequence point falls in the high risk area, a longitudinal collision avoidance operation is performed.
If the transverse collision avoidance operation is determined, the transverse planning layer plans a transverse collision avoidance path according to the state of the pedestrians relative to the vehicle; if the longitudinal collision avoidance operation is determined, an emergency braking or deceleration method is adopted to achieve the purpose of the longitudinal collision avoidance operation.
More specifically, the specific steps of the lateral planning layer for planning the lateral collision avoidance path are as follows:
(1) Combining the pedestrian track predicted by the S3 or the S4, and dividing a transverse collision avoidance path by adopting an artificial potential field rule; constructing an attractive potential field, a road boundary repulsive potential field and an elliptical obstacle repulsive potential field:
(2) And (3) carrying out negative gradient on the attraction potential field, the road boundary repulsive potential field and the elliptical obstacle repulsive potential field to obtain potential field force corresponding to each potential field, wherein the potential field force is as follows:
adding the three potential field forces to obtain the resultant force born by the vehicle:
Ftotal＝Falt+Froad+Fobs
Wherein U alt represents a gravitational potential field to which the vehicle is subjected, U road represents a road boundary repulsive potential field to which the vehicle is subjected, U obs represents an obstacle repulsive potential field to which the vehicle is subjected, F alt represents gravitational force, F road represents a road boundary repulsive force, F obs represents an obstacle repulsive force, K alt represents a gravitational potential field gain coefficient, X represents a vehicle real-time coordinate, X g represents a vehicle target point coordinate, K road represents a road boundary constraint coefficient, X represents a vehicle coordinate in an X-direction, y represents a vehicle coordinate in a y-axis direction, y road,i represents an ordinate of an i-th road boundary line, W represents a vehicle width, (X obs,yobs) represents an obstacle coordinate, sigma x and sigma y represent distance influencing factors by which the obstacle acts on the vehicle, For representing gradient calculations;
(3) The position points where the vehicle moves under the action of the force of the combined force can be obtained by the force balance of the potential field force in the transverse direction, and the transverse planning path of obstacle avoidance can be obtained by curve fitting of the points;
(4) According to the position change in the process of crossing the pedestrian, carrying out collision risk analysis on the vehicle and the pedestrian in real time, and judging whether a path is required to be re-planned or not, so as to obtain a real-time transverse collision avoidance path; in this embodiment, the collision risk analysis is performed by the vehicle and the pedestrian according to whether the position area where the pedestrian may appear and the position area where the vehicle may appear overlap in the planning period, if so, it is considered that there is a collision risk that the path needs to be re-planned, otherwise, it is considered that there is no collision risk and the path does not need to be re-planned.
More specifically, the specific steps of the emergency braking or decelerating method are as follows:
(1) If emergency braking is adopted, a fuzzy controller is established, the relative speed and the relative distance between the vehicle and the pedestrian are used as input, and the expected deceleration is output;
(2) If the pedestrian is decelerated to avoid and normally runs after passing through the road, the expected minimum deceleration of the vehicle is obtained by the following formula:
Wherein:
Where t pass denotes a time for a pedestrian to traverse a road, L path denotes a length of the road, Y p denotes coordinates of the pedestrian in the Y direction, v p-y denotes a speed of the pedestrian in the Y direction, v vehicle denotes a speed of the vehicle, v ped denotes a speed of the pedestrian, Δs denotes a longitudinal relative distance between the pedestrian and the vehicle, and a denotes a desired minimum deceleration.
In order to realize the pedestrian track prediction and vehicle active collision avoidance method considering the human-vehicle interaction, the application also designs a pedestrian track prediction and vehicle active collision avoidance system considering the human-vehicle interaction shown in the figure 1, and the system specifically comprises an environment sensing module, a pedestrian intention judging module, a pedestrian track prediction module and a longitudinal and transverse collision avoidance decision module.
The environment sensing module is used for acquiring the running state information (vehicle speed v vehicle and vehicle position), the pedestrian movement state information (pedestrian speed v ped and pedestrian position) and the vehicle front image, and is respectively connected with the pedestrian intention judging module, the pedestrian track predicting module and the longitudinal and transverse collision avoidance decision module. The environment sensing module specifically comprises a GPS, a speed sensor, a laser radar and a monocular camera which are arranged on a vehicle, position information and speed information of the vehicle, position information and speed information of pedestrians crossing the street are obtained in real time, and a Distance between the vehicle and the vehicle is obtained based on the position information of the vehicle and the position information of the pedestrians crossing the street; the front image of the vehicle is acquired by a monocular camera.
The pedestrian intention judging module detects the head of a pedestrian in the front image of the vehicle by utilizing the image processing unit according to the running state information, the pedestrian movement state information and the front image of the vehicle, acquired by the environment sensing module, so that the face orientation of the pedestrian crossing the street is identified, and when the face of the pedestrian faces the coming vehicle, the pedestrian is considered to notice the coming vehicle; otherwise, pedestrians are not considered to notice the coming vehicle.
Judging the intention of the pedestrian crossing based on the recognition result of the face orientation of the pedestrian in the pedestrian intention judging module, and sending the intention judging result to the pedestrian track predicting module; the specific process of pedestrian intention judgment is as follows:
(1) The pedestrian speed v ped, the pedestrian face Orientation degree, the vehicle speed v vehicle and the human-vehicle Distance are taken as independent variables, the intention of the pedestrian to select to walk or stop when facing an incoming vehicle in the human-vehicle interaction process is taken as the dependent variable, and the function expression in the logistic regression model for judging the pedestrian intention is as follows:
Wherein, the independent variable x= [ c, v ped,Orientation,vvehicle,Distance]T, c is any constant term. θ= [ θ 0,θ1,θ2,θ3,θ4]T ] is a coefficient set, θ i is a coefficient corresponding to the i-th argument, i=0, 1,2, 3, 4. In this embodiment, the coefficient set θ is obtained by obtaining the minimum value of the cost function by the gradient descent method. The expression formula of the cost function is as follows:
Where J (θ) is the cost function, m is the number of samples, H θ is the hypothesized value, y (i) is the ith real value, x (i) is the ith argument, x (i) ε x.
(2) And inputting the actually detected x= [ c, v ped,Orientation,vvehicle,Distance]T into a trained logistic regression model to judge the intention of the pedestrian, wherein if H θ is more than 0.5, H θ is less than 0.5, the pedestrian stops.
The pedestrian track prediction module receives the running state information of the vehicle, the moving state information of the pedestrians and the judging result of the pedestrian intention judging module, which are sent by the environment sensing module, and predicts the track of the pedestrians in the future preset time length by using a Markov pedestrian model for the pedestrians which do not notice the vehicle; for pedestrians who notice the vehicle and continue to walk (i.e., H θ > 0.5), the motion of the pedestrian across the street is considered a combination of free motion and the motion of the oncoming vehicle disturbance. Introducing a social force model into the motion prediction of the pedestrians at the moment, and carrying out weighted fusion on the pedestrian position predicted by the prior Markov pedestrian model and the pedestrian position predicted by the social force model to obtain a corrected pedestrian position; and obtaining a track curve in a preset time length according to the set time step length, and sending the predicted track curve to a longitudinal and transverse collision avoidance decision module.
Within the pedestrian trajectory prediction module, for a pedestrian not noticing a vehicle, the future position and speed of the pedestrian depends on his current position and speed, from which a state description of the pedestrian can be derived:
Stateped＝(xp(t),yp(t),vx-p(t),vy-p(t))
Wherein State ped is the pedestrian State; v x-p (t) and v y-p (t) represent the speeds of the pedestrian at the time t in the X-direction and the Y-direction, respectively, Δv x-p and Δv y-p represent the speed increments at the time t in the X-direction and the Y-direction, respectively, v x-p (t+Δt) and v y-p (t+Δt) represent the speeds of the pedestrian at the time t+Δt in the X-direction and the Y-direction, respectively, k x and k y represent constants, AndRepresenting the expected speed of the pedestrian in the X direction and the Y direction respectively, epsilon x and epsilon y representing the random disturbance of the speed increment of the pedestrian in the X direction and the Y direction respectively, X p (t) and Y p (t) representing the displacement function of the pedestrian in the X direction and the Y direction relative to time t respectively, and p m (t) representing the position of the pedestrian at t predicted by the Markov pedestrian model.
The pedestrian notices the vehicle and is intended to walk in the face of the incoming vehicle, and the motion prediction of the pedestrian at this time introduces a social force model, comprising the following specific steps:
The pedestrian moves to the target point and has a driving force, the pedestrian can be subjected to the repulsive force of the vehicle facing the incoming vehicle, the road can also apply a hidden boundary force to the pedestrian, the forces are accumulated to form a resultant force, and the position of the pedestrian is recursively calculated along with the time step under the action of the social force. The expression formula of the resultant force and the position of the pedestrian is as follows:
Where F ped (t) represents the resultant social force applied to the pedestrian, F d represents the driving force applied to the pedestrian to move toward the target point, F vp represents the repulsive force applied to the pedestrian facing the incoming vehicle, and F e represents the road to apply a hidden boundary force to the pedestrian. v n (t) represents the speed of the pedestrian at t, p s (t) represents the position of the pedestrian at t predicted by the social force model, p s (t+Δt) represents the position of the pedestrian at t+Δt, Δt represents the time step, and m represents the mass of the pedestrian.
Regarding pedestrians which pay attention to vehicles and continue to walk, regarding the movement of pedestrians passing through the street as a combination of free movement and interference movement of coming vehicles, carrying out weighted fusion on the positions of the pedestrians predicted by the Markov pedestrian model and the positions of the pedestrians predicted by the social force model, and obtaining corrected positions of the pedestrians, wherein the corrected positions are represented by the following formula:
p(t)＝τm·pm(t)+τs·ps(t)
Where p (t) represents the pedestrian position at t obtained by fusing and correcting the markov pedestrian model and the social force model, τ m and τ s represent the weight coefficients of p m (t) and p s (t), respectively.
And respectively obtaining a predicted track curve of the pedestrian which does not notice the vehicle and a predicted track curve of the pedestrian which notices the vehicle and continues to walk in the preset time length according to the set time step.
The longitudinal and transverse collision avoidance decision module receives the pedestrian track predicted by the pedestrian track prediction module, evaluates the feasibility of longitudinal collision avoidance and transverse collision avoidance through the safety state analysis of the pedestrians and the vehicles, and decides a proper collision avoidance strategy of the vehicles under the condition of ensuring the safety. The longitudinal and transverse collision avoidance selection strategy in the longitudinal and transverse collision avoidance decision module comprises the following steps:
(1) Dividing the position of pedestrians on a road into a dangerous area, a high-risk area and a safe area; FIG. 5 is a division of pedestrian hazard areas, with the area from the right edge of the vehicle to the right edge of the road being the hazard area; the area directly in front of the vehicle may be regarded as a high risk area; other areas on the road may be considered safe areas.
(2) And calculating the time for the longitudinal collision avoidance of the vehicle. Considering the position fluctuation of the pedestrian position in the longitudinal direction, the obtained longitudinal collision avoidance time of the vehicle is a time zone range:
Where t v denotes a vehicle longitudinal collision avoidance time, v vehicle denotes a vehicle speed, Δs denotes a longitudinal relative distance between the vehicle and the pedestrian, ε denotes a longitudinal offset when the pedestrian crosses the street, and κ denotes a time elastic factor.
(3) Uniformly sampling the obtained time zone segment to obtain a series of time sequence points:
{t-κ,t-κ+Δt’,t-κ+2Δt’,…,t-κ+(n-1)Δt’,t+κ}
Where Δt' is the time interval of uniform sampling.
(4) Substituting the time sequence points into a position expression of the predicted track to generate a position sequence in the time region:
{Pt-κ,Pt-κ+Δt',Pt-κ+2Δt',…,Pt+κ-Δt',Pt+κ}
(5) Deciding a collision avoidance strategy of the vehicle according to the number of the position sequence points in the three areas (dangerous area, high risk area and safe area) obtained by dividing:
1) If all the sequence points fall in the dangerous area, performing transverse collision avoidance operation;
2) If all the sequence points fall in the high risk area, performing longitudinal collision avoidance operation;
3) As long as one sequence point falls in the high risk area, a longitudinal collision avoidance operation is performed.
If the transverse collision avoidance operation is determined, the transverse planning layer plans a transverse collision avoidance path according to the state of the pedestrians relative to the vehicle; if the longitudinal collision avoidance operation is determined, an emergency braking or deceleration method is adopted to achieve the purpose of the longitudinal collision avoidance operation.
More specifically, the specific steps of the lateral planning layer for planning the lateral collision avoidance path are as follows:
(1) Combining the pedestrian track predicted by the S3 or the S4, and dividing a transverse collision avoidance path by adopting an artificial potential field rule; constructing an attractive potential field, a road boundary repulsive potential field and an elliptical obstacle repulsive potential field:
(2) And (3) carrying out negative gradient on the attraction potential field, the road boundary repulsive potential field and the elliptical obstacle repulsive potential field to obtain potential field force corresponding to each potential field, wherein the potential field force is as follows:
adding the three potential field forces to obtain the resultant force born by the vehicle:
Ftotal＝Falt+Froad+Fobs
Wherein U alt represents a gravitational potential field to which the vehicle is subjected, U road represents a road boundary repulsive potential field to which the vehicle is subjected, U obs represents an obstacle repulsive potential field to which the vehicle is subjected, F alt represents gravitational force, F road represents a road boundary repulsive force, F obs represents an obstacle repulsive force, K alt represents a gravitational potential field gain coefficient, X represents a vehicle real-time coordinate, X g represents a vehicle target point coordinate, K road represents a road boundary constraint coefficient, X represents a vehicle coordinate in an X-direction, y represents a vehicle coordinate in a y-axis direction, y road,i represents an ordinate of an i-th road boundary line, W represents a vehicle width, (X obs,yobs) represents an obstacle coordinate, sigma x and sigma y represent distance influencing factors by which the obstacle acts on the vehicle, For representing gradient calculations;
(3) The position points where the vehicle moves under the action of the force of the combined force can be obtained by the force balance of the potential field force in the transverse direction, and the transverse planning path of obstacle avoidance can be obtained by curve fitting of the points;
(4) According to the position change in the pedestrian crossing process, collision risk analysis is carried out on the vehicles and pedestrians in real time, and whether a path is required to be re-planned is judged, so that a real-time transverse collision avoidance path is obtained. In this embodiment, the collision risk analysis is performed by the vehicle and the pedestrian according to whether the position area where the pedestrian may appear and the position area where the vehicle may appear overlap in the planning period, if so, it is considered that there is a collision risk that the path needs to be re-planned, otherwise, it is considered that there is no collision risk and the path does not need to be re-planned.
More specifically, the specific steps of the emergency braking or decelerating method are as follows:
(1) If emergency braking is adopted, a fuzzy controller is established, the relative speed and the relative distance between the vehicle and the pedestrian are used as input, and the expected deceleration is output;
(2) If the pedestrian is decelerated to avoid and normally runs after passing through the road, the expected minimum deceleration of the vehicle is obtained by the following formula:
Wherein:
Where t pass denotes a time for a pedestrian to traverse a road, L path denotes a length of the road, Y p denotes coordinates of the pedestrian in the Y direction, v p-y denotes a speed of the pedestrian in the Y direction, v vehicle denotes a speed of the vehicle, v ped denotes a speed of the pedestrian, Δs denotes a longitudinal relative distance between the pedestrian and the vehicle, and a denotes a desired minimum deceleration.
The above embodiments are merely for illustrating the design concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement the same, the scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications according to the principles and design ideas of the present invention are within the scope of the present invention.

Claims (8)

1. A pedestrian track prediction and active collision avoidance method considering human-vehicle interaction is characterized by comprising the following steps:
s1, acquiring running state information of a vehicle, pedestrian movement state information and a front image of the vehicle;
S2, extracting a pedestrian head area and identifying the face direction of the pedestrian according to the acquired front image of the vehicle, and judging whether the pedestrian notices an incoming vehicle or not according to the face direction of the pedestrian; based on a logistic regression model, fusing pedestrian motion state information, pedestrian face orientation and vehicle motion state information, and judging the intention of a pedestrian;
s3, according to the face orientation of the pedestrian, if the pedestrian is judged to not notice the vehicle, predicting the track of the pedestrian in the future preset duration by using a Markov pedestrian model; the process of predicting the track of the pedestrian in the future preset time length by using the Markov pedestrian model is as follows:
The pedestrian does not notice the vehicle, at this time, the vehicle on the road can be regarded as no interference to the pedestrian, the free motion of the pedestrian passing through the street under the condition of no external interference accords with the Markov process, the future position and speed of the pedestrian depend on the current position and speed of the pedestrian, and the state description of the pedestrian can be obtained:
Stateped＝(xp(t),yp(t),vx-p(t),vy-p(t))
Where State ped is the State of the pedestrian, v x-p (t) and v y-p (t) represent the speeds of the pedestrian in the X-direction and Y-direction at time t, respectively, deltav x-p and Deltav y-p represent the speed increment in the X-direction and Y-direction at time t, v x-p (t+Deltat) and v y-p (t+Deltat) represent the speeds of the pedestrian in the X-direction and Y-direction at+Deltat, respectively, k x and k y represent constants, AndRepresenting expected speeds of pedestrians in the X direction and the Y direction respectively, epsilon x and epsilon y representing random disturbance of speed increment of the pedestrians in the X direction and the Y direction respectively, X p (t) and Y p (t) representing displacement functions of the pedestrians in the X direction and the Y direction relative to time t respectively, and p m (t) representing positions of the pedestrians at t predicted by the Markov pedestrian model;
S4, according to the face direction of the pedestrian and the pedestrian intention judging result, if the pedestrian is judged to notice the vehicle and continue to walk, a social force model is introduced to predict the motion of the pedestrian at the moment; introducing a Markov pedestrian model to predict in the pedestrian movement process; the pedestrian position predicted by the Markov pedestrian model and the pedestrian position predicted by the social force model are subjected to weighted fusion to obtain a corrected pedestrian position; obtaining a track curve in a preset duration according to the set time step; predicting the motion of the pedestrian by using the social force model, wherein the method specifically comprises the following steps:
The pedestrian moves to the target point to have a driving force, the pedestrian can be subjected to the repulsive force of the vehicle facing the coming vehicle, the road can also apply a hidden boundary force to the pedestrian, the forces are accumulated to form a resultant force, and the position of the pedestrian is recursively calculated along with the time step under the action of the social force; the expression formula of the resultant force and the position of the pedestrian is as follows:
Wherein F ped (t) represents the resultant force of social force received by the pedestrian, F d represents the driving force received by the pedestrian when moving towards the target point, F vp represents the repulsive force received by the pedestrian when facing the incoming vehicle, F e represents the road to apply a hidden boundary force to the pedestrian, v n (t) represents the speed of the pedestrian at t, p s (t) represents the position of the pedestrian predicted by the social force model at t, p s (t+Δt) represents the position of the pedestrian at t+Δt, Δt represents the time step, and m represents the mass of the pedestrian;
S5, judging the safety state according to the predicted pedestrian track, and deciding a proper collision avoidance strategy of the vehicle under the condition of ensuring safety;
In the above S2, the pedestrian face orientation recognition step includes:
(1) Detecting the head area of the pedestrian by adopting Yolo algorithm according to the front image of the vehicle to obtain a head image;
(2) Constructing a forward/lateral/back classifier of the face of the pedestrian by adopting a convolutional neural network, and determining the Orientation of the face of the pedestrian;
in the above S2, the logistic regression model for pedestrian intention judgment includes the following steps:
(1) The pedestrian speed v ped, the pedestrian face Orientation degree, the vehicle speed v vehicle and the human-vehicle Distance are taken as independent variables, the intention of the pedestrian to select to walk or stop when facing an incoming vehicle in the human-vehicle interaction process is taken as the dependent variable, and the function expression in the logistic regression model for judging the pedestrian intention is as follows:
Wherein, the independent variable is expressed as x= [ c, v ped,Orientation,vvehicle,Distance]T, c is any constant term; θ= [ θ 0,θ1,θ2,θ3,θ4]T ] is a coefficient set, θ i is a coefficient corresponding to the i-th argument, i=0, 1, 2, 3, 4; obtaining the coefficient corresponding to the ith independent variable as a minimum value theta i of the cost function through a gradient descent method; the expression formula of the cost function is as follows:
Wherein J (θ) is a cost function, m is the number of samples, H θ is a hypothetical value, y (i) is the ith real value, x (i) is the ith argument, x (i) ε x;
(2) And judging the intention of the pedestrian by using the trained logistic regression model, wherein if H θ is more than 0.5, H θ is less than 0.5, the pedestrian stops.
2. The method for predicting and actively avoiding collision according to claim 1, wherein the motion of the pedestrian passing through the street is considered as a combination of free non-interference motion and interference motion of the coming vehicle, and the predicted pedestrian position is obtained by weighting the pedestrian position predicted by the markov pedestrian model and the pedestrian position predicted by the social force model, wherein the pedestrian position is represented by the following formula:
p(t)＝τm·pm(t)+τs·ps(t)
Where p (t) represents the pedestrian position at t obtained by fusing and correcting the markov pedestrian model and the social force model, τ m and τ s represent the weight coefficients of p m (t) and p s (t), respectively.
3. The method for predicting and actively avoiding collision according to claim 1, wherein in S5, the method for determining the safety state of the pedestrian and the vehicle is as follows:
(1) Dividing the position of pedestrians on a road into a dangerous area, a high-risk area and a safe area;
(2) Calculating the longitudinal collision avoidance time of the vehicle, and considering the position fluctuation of the pedestrian position in the longitudinal direction, wherein the obtained longitudinal collision avoidance time of the vehicle is a time region range:
Wherein t v represents the longitudinal collision avoidance time of the vehicle, v vehicle represents the vehicle speed, Δs represents the longitudinal relative distance between the vehicle and the pedestrian, ε represents the longitudinal offset when the pedestrian crosses the street, and κ represents the time elasticity factor;
(3) Uniformly sampling the obtained time zone segment to obtain a series of time sequence points:
{t-κ,t-κ+Δt’,t-κ+2Δt’,…,t-κ+(n-1)Δt’,t+κ}
wherein Δt' is the time interval of uniform sampling;
(4) Substituting the time sequence points into a position expression of the predicted track to generate a position sequence in the time region:
{Pt-κ,Pt-κ+Δt',Pt-κ+2Δt',…,Pt+κ-Δt',Pt+κ}
(5) And deciding a collision avoidance strategy of the vehicle according to the number of the position sequence points in the three areas obtained by dividing.
4. The method for predicting and actively avoiding collision according to claim 3, wherein the collision avoidance strategy of the vehicle is as follows:
1) If all the sequence points fall in the dangerous area, performing transverse collision avoidance operation;
2) If all the sequence points fall in the high risk area, performing longitudinal collision avoidance operation;
3) As long as one sequence point falls in a high risk area, performing longitudinal collision avoidance operation;
If the transverse collision avoidance operation is determined, the transverse planning layer plans a transverse collision avoidance path according to the state of the pedestrians relative to the vehicle; if the longitudinal collision avoidance operation is determined, an emergency braking or deceleration method is adopted to achieve the purpose of the longitudinal collision avoidance operation.
5. The method for predicting and actively preventing collision of pedestrian trajectories with consideration of human-vehicle interaction according to claim 4, wherein the specific step of planning the lateral collision prevention path by the lateral planning layer is as follows:
(1) Combining the pedestrian track predicted by the S3 or the S4, and dividing a transverse collision avoidance path by adopting an artificial potential field rule; constructing an attractive potential field, a road boundary repulsive potential field and an elliptical obstacle repulsive potential field:
(2) And (3) carrying out negative gradient on the attraction potential field, the road boundary repulsive potential field and the elliptical obstacle repulsive potential field to obtain potential field force corresponding to each potential field, wherein the potential field force is as follows:
adding the three potential field forces to obtain the resultant force born by the vehicle:
Ftotal＝Falt+Froad+Fobs
Wherein U alt represents a gravitational potential field to which the vehicle is subjected, U road represents a road boundary repulsive potential field to which the vehicle is subjected, U obs represents an obstacle repulsive potential field to which the vehicle is subjected, F alt represents gravitational force, F road represents a road boundary repulsive force, F obs represents an obstacle repulsive force, K alt represents a gravitational potential field gain coefficient, X represents a vehicle real-time coordinate, X g represents a vehicle target point coordinate, K road represents a road boundary constraint coefficient, X represents a vehicle coordinate in an X-direction, y represents a vehicle coordinate in a y-axis direction, y road,i represents an ordinate of an i-th road boundary line, W represents a vehicle width, (X obs,yobs) represents an obstacle coordinate, sigma x and sigma y represent distance influencing factors by which the obstacle acts on the vehicle, For representing gradient calculations;
(3) The position points where the vehicle moves under the action of the force of the combined force can be obtained by the force balance of the potential field force in the transverse direction, and the transverse planning path of obstacle avoidance can be obtained by curve fitting of the points;
(4) According to the position change in the process of crossing the pedestrian, carrying out collision risk analysis on the vehicle and the pedestrian in real time, and judging whether a path is required to be re-planned or not, so as to obtain a real-time transverse collision avoidance path; the collision risk analysis is performed by the vehicle and the pedestrian according to whether the position area where the pedestrian may appear and the position area where the vehicle may appear overlap in the planning period, if so, the collision risk is considered to exist, and the path is required to be re-planned, otherwise, the collision risk is considered not to exist, and the path is not required to be re-planned.
6. The method for predicting and actively avoiding collision according to claim 4, wherein the specific steps of adopting an emergency braking or decelerating method are as follows:
(1) If emergency braking is adopted, a fuzzy controller is established, the relative speed and the relative distance between the vehicle and the pedestrian are used as input, and the expected deceleration is output;
(2) If the pedestrian is decelerated to avoid and normally runs after passing through the road, the expected minimum deceleration of the vehicle is obtained by the following formula:
Wherein:
Where t pass denotes a time for a pedestrian to traverse a road, L path denotes a length of the road, Y p denotes coordinates of the pedestrian in the Y direction, v p-y denotes a speed of the pedestrian in the Y direction, v vehicle denotes a speed of the vehicle, v ped denotes a speed of the pedestrian, Δs denotes a longitudinal relative distance between the pedestrian and the vehicle, and a denotes a desired minimum deceleration.
7. A pedestrian track prediction and active collision avoidance system considering human-vehicle interaction, which is characterized in that the system adopts the pedestrian track prediction and active collision avoidance method considering human-vehicle interaction according to any one of claims 1-6, and comprises an environment sensing module, a pedestrian intention judging module, a pedestrian track prediction module and a longitudinal and transverse collision avoidance decision module;
the environment sensing module is used for acquiring the running state information of the vehicle, the moving state information of the pedestrians and the front image of the vehicle, and is respectively connected with the pedestrian intention judging module, the pedestrian track predicting module and the longitudinal and transverse collision avoidance decision module;
The pedestrian intention judging module detects the head of a pedestrian in the front image of the vehicle by utilizing the image processing unit according to the running state information, the pedestrian movement state information and the front image of the vehicle, which are acquired by the environment sensing module, so as to recognize the face orientation of the pedestrian crossing the street; judging the intention of the pedestrian crossing based on the recognition result of the face orientation of the pedestrian, and sending the intention judgment result to a pedestrian track prediction module;
The pedestrian track prediction module receives the running state information of the vehicle, the moving state information of the pedestrians and the judging result of the pedestrian intention judging module, which are sent by the environment sensing module, and predicts the track of the pedestrians in the future preset time length by using a Markov pedestrian model for the pedestrians which do not notice the vehicle; regarding pedestrians who notice vehicles and continue to walk, regarding the movement of pedestrians crossing the street as a combination of free movement and interference movement of coming vehicles; introducing a social force model into the motion prediction of the pedestrians at the moment, and carrying out weighted fusion on the pedestrian position predicted by the prior Markov pedestrian model and the pedestrian position predicted by the social force model to obtain a corrected pedestrian position; obtaining a track curve in a preset time length according to a set time step, and sending the predicted track curve to a longitudinal and transverse collision avoidance decision module;
the longitudinal and transverse collision avoidance decision module receives the pedestrian track predicted by the pedestrian track prediction module, evaluates the feasibility of longitudinal collision avoidance and transverse collision avoidance through the safety state analysis of the pedestrians and the vehicles, and decides a proper collision avoidance strategy of the vehicles under the condition of ensuring the safety.
8. The system for predicting the trajectory of pedestrians and actively avoiding collisions according to claim 7, wherein the environmental awareness module comprises a GPS, a speed sensor, a laser radar and a monocular camera which are installed on the vehicle, and the system is used for obtaining the position information and the speed information of the vehicle, the position information and the speed information of the pedestrians crossing the street relative to the vehicle in real time; the monocular camera acquires an image in front of the vehicle.
CN202111669198.9A 2021-12-31 2021-12-31 Pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction Active CN114371707B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111669198.9A CN114371707B (en) 2021-12-31 2021-12-31 Pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111669198.9A CN114371707B (en) 2021-12-31 2021-12-31 Pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction

Publications (2)

Publication Number Publication Date
CN114371707A CN114371707A (en) 2022-04-19
CN114371707B true CN114371707B (en) 2024-07-12

Family

ID=81141560

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111669198.9A Active CN114371707B (en) 2021-12-31 2021-12-31 Pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction

Country Status (1)

Country Link
CN (1) CN114371707B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114926422B (en) * 2022-05-11 2023-07-04 西南交通大学 Method and system for detecting passenger flow of getting on and off vehicles
US20240092359A1 (en) * 2022-09-16 2024-03-21 GM Global Technology Operations LLC Vehicle camera-based prediction of change in pedestrian motion
CN115628736B (en) * 2022-09-23 2024-10-11 北京智行者科技股份有限公司 Pedestrian track prediction method, equipment, mobile device and storage medium
CN117037500B (en) * 2023-10-10 2023-12-22 蘑菇车联信息科技有限公司 Method, equipment and medium for predicting small target track under unstructured road
CN118212783B (en) * 2024-05-21 2024-08-06 南京邮电大学 Pedestrian crossing track prediction method and device for signalized intersection

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110414365A (en) * 2019-07-03 2019-11-05 上海交通大学 Street pedestrian's trajectory predictions method, system and medium based on social force model
CN111429754A (en) * 2020-03-13 2020-07-17 南京航空航天大学 Vehicle collision avoidance track risk assessment method under pedestrian crossing working condition

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106945668B (en) * 2016-10-27 2019-08-16 蔚来汽车有限公司 Vehicle driving arrow path auxiliary system
US10994729B2 (en) * 2017-03-29 2021-05-04 Mitsubishi Electric Research Laboratories, Inc. System and method for controlling lateral motion of vehicle
CN113720346B (en) * 2021-09-02 2023-07-04 重庆邮电大学 Vehicle path planning method and system based on potential energy field and hidden Markov model

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110414365A (en) * 2019-07-03 2019-11-05 上海交通大学 Street pedestrian's trajectory predictions method, system and medium based on social force model
CN111429754A (en) * 2020-03-13 2020-07-17 南京航空航天大学 Vehicle collision avoidance track risk assessment method under pedestrian crossing working condition

Also Published As

Publication number Publication date
CN114371707A (en) 2022-04-19

Similar Documents

Publication Publication Date Title
CN114371707B (en) Pedestrian track prediction and active collision avoidance method and system considering human-vehicle interaction
CN110488802B (en) Decision-making method for dynamic behaviors of automatic driving vehicle in internet environment
Do et al. Human drivers based active-passive model for automated lane change
CN110705388B (en) Target vehicle lane change identification method for auxiliary driving based on prediction feedback
US8947218B2 (en) Driving support device
CN110843789B (en) Vehicle lane change intention prediction method based on time sequence convolution network
US8788134B1 (en) Autonomous driving merge management system
CN110155046A (en) Automatic emergency brake hierarchical control method and system
CN112435503B (en) Intelligent automobile active collision avoidance method for identifying intention of high-risk pedestrians
CN110077398B (en) Risk handling method for intelligent driving
CN105809130A (en) Binocular depth perception-based vehicle travelable area calculation method
CN112292717A (en) Prediction control device
CN114987539A (en) Individual collision grading early warning method and system for automatic driving automobile based on risk field model
Guo et al. Toward human-like behavior generation in urban environment based on Markov decision process with hybrid potential maps
CN115909783A (en) Lane-level driving assistance method and system based on traffic flow
Yao et al. Target vehicle selection algorithm for adaptive cruise control based on lane-changing intention of preceding vehicle
CN116654017A (en) Intelligent automobile collision avoidance decision and path planning method and system based on peripheral automobile track prediction under emergency working condition
CN114932918A (en) Behavior decision method and system for intelligent internet vehicle to drive under various road conditions
CN117529429A (en) Method and system for predicting behavior of an actor in an autonomous vehicle environment
CN114446046A (en) LSTM model-based weak traffic participant track prediction method
CN113159576A (en) Driving risk calculation method and online evaluation system of automatic driving automobile
CN113460083A (en) Vehicle control device, vehicle control method, and storage medium
US20230227082A1 (en) Vehicle speed management systems and methods
JP6796679B2 (en) Vehicle control system and method, and driving support server
CN115140048B (en) Automatic driving behavior decision and trajectory planning model and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20240509

Address after: 518000 1104, Building A, Zhiyun Industrial Park, No. 13, Huaxing Road, Henglang Community, Longhua District, Shenzhen, Guangdong Province

Applicant after: Shenzhen Hongyue Enterprise Management Consulting Co.,Ltd.

Country or region after: China

Address before: Zhenjiang City, Jiangsu Province, 212013 Jingkou District Road No. 301

Applicant before: JIANGSU University

Country or region before: China

TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20240617

Address after: Building 6, 7th Floor, Artificial Intelligence Industrial Park, No. 266 Chuangyan Road, Qilin Technology Innovation Park, Nanjing City, Jiangsu Province, 211135

Applicant after: NANJING KINGYOUNG INTELLIGENT SCIENCE AND TECHNOLOGY Co.,Ltd.

Country or region after: China

Address before: 518000 1104, Building A, Zhiyun Industrial Park, No. 13, Huaxing Road, Henglang Community, Longhua District, Shenzhen, Guangdong Province

Applicant before: Shenzhen Hongyue Enterprise Management Consulting Co.,Ltd.

Country or region before: China

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant