CN114228690B

CN114228690B - Automatic driving vehicle roll control method based on DDPG and iterative control

Info

Publication number: CN114228690B
Application number: CN202111353270.7A
Authority: CN
Inventors: 唐晓峰
Original assignee: Yangzhou University
Current assignee: Yangzhou University
Priority date: 2021-11-16
Filing date: 2021-11-16
Publication date: 2023-05-23
Anticipated expiration: 2041-11-16
Also published as: CN114228690A

Abstract

The invention discloses an automatic driving vehicle roll control method based on DDPG and iterative control, which trains a DDPG algorithm on running maps of the automatic driving vehicle in different scenes, and the automatic driving vehicle generates real-time vehicle states through interaction with map environments in different scenes to determine the action behaviors of the vehicle; initializing an action space when action training is carried out, generating state space information by an online strategy network in an actor network, carrying out action output, and adding an action noise to obtain the action space with exploratory property; based on LSTM historic memory and road planning attributes, generating a predicted path of the state of the automatic driving vehicle, adopting a DDPG algorithm to realize tracking control of the path track under the normal running working condition and the extreme running working condition of the automatic driving vehicle, and adopting an iterative control method to realize compensation control of the automatic driving vehicle. The invention avoids the instability problem of the reinforcement learning algorithm under the extreme road environment driving condition of the vehicle, and improves the driving safety and the robustness of the vehicle.

Description

Automatic driving vehicle roll control method based on DDPG and iterative control

Technical Field

The invention belongs to the field of intelligent vehicle control, and particularly relates to an automatic driving vehicle roll control method based on DDPG and iterative control.

Background

With the development of artificial intelligence technology, the current automatic driving technology has greatly developed, and is currently focused on closed campus scenes, such as closed campus scenes, logistics industry park scenes and the like, and particularly, the application is more common in harbour road environments with fewer structured road features, pedestrians and vehicles. The automatic driving vehicle adopts environment sensing, navigation, map positioning, decision making, motion planning and track tracking control to realize the intellectualization of the vehicle. However, when an automatic driving vehicle is in complex weather and complex driving environments such as a cross-sea bridge, the severe weather environment can influence the road condition of the bridge, so that the vehicle turns, sideslips and turns over, such as rain, snow, wind and the like in the weather environment, the road attachment coefficient is changed, tires are slipped, and the path tracking, lane keeping and vehicle control precision are changed. In addition, the bridge road environment may vibrate due to the influence of wind weather, and a roll phenomenon of a vehicle may occur, thereby causing an uncontrollable situation. Therefore, the control technique is a complex task when the vehicle has uncertainty characteristics such as a sideslip phenomenon caused by a wet road surface, a vehicle yaw characteristic caused by bridge vibration, a vehicle roll dynamics phenomenon caused by high-speed vehicle performance, and the like, also considering road vibration characteristics, road angle, and aerodynamic characteristics in the overall control design. Therefore, running safety and stability research based on the roll control of an automatic driving vehicle under severe weather conditions is an important key technology. Reinforcement learning is an application category of artificial intelligence technology, and an intelligent agent can explore an unknown dynamic environment, try different actions and interact with the dynamic environment without any accurate vehicle model and given surrounding environment, can learn the unknown environment, realize complex vehicle dynamics through actions and states interacted with the environment, and is a new implementation method for adapting to dynamic road environment cognition and complex vehicle dynamics performance. Therefore, the adoption of the reinforcement learning algorithm to realize the control of the vehicle in the running environment of the automatic driving vehicle on the road condition of the cross-sea bridge is beneficial to realizing the intelligent safety of the vehicle and the scale industrialized development of the automatic driving vehicle.

Disclosure of Invention

The invention aims to: the invention provides an automatic driving vehicle roll control method based on DDPG and iterative control, which enables a vehicle to safely run in cross-sea bridge crossing environments with any different complexity levels and improves the intelligent level of the vehicle through instantaneous roll angles.

The technical scheme is as follows: the invention provides an automatic driving vehicle roll control method based on DDPG and iterative control, which specifically comprises the following steps:

(1) Installing a laser radar, a vision, a millimeter wave radar, an ultrasonic radar sensor, a positioning system and an inertial navigation system on an autopilot vehicle;

(2) The method comprises the steps of using a visual sensor, a positioning system and an inertial navigation system to realize positions and maps of vehicles in different scenes respectively so as to generate automatic driving vehicle running maps in different scenes and realize the environment required by vehicle running tracks;

(3) Respectively controlling a steering wheel, an accelerator and a pedal, driving on a cross-sea bridge, acquiring corresponding driving tracks in rainy and snowy days, strong wind and bad weather and sunny days, and constructing a data set;

(4) The DDPG algorithm is trained on the running map of the automatic driving vehicle under different scenes and is used for running states of the cross-sea bridge under different complex road condition grades in severe weather; the automatic driving vehicle generates real-time vehicle states through interaction with map environments in different scenes, and determines action behaviors of the vehicle; initializing an action space when action training is carried out, generating state space information by an online strategy network in an actor network, carrying out action output, and adding an action noise to obtain the action space with exploratory property;

(5) Based on LSTM historic memory and road planning attributes, generating a predicted path of the state of the automatic driving vehicle, adopting a DDPG algorithm to realize tracking control of the path track of the automatic driving vehicle under normal driving road conditions and extreme driving road conditions, and adopting an iterative control method to realize compensation control of the automatic driving vehicle.

Further, the lidar sensor in step (1) is used to detect dynamic and static obstacles on roads, including pedestrians, motorcycles, various vehicles, etc., and movable road areas; the visual sensor is used for sensing lane lines, pedestrians and vehicles and performing positioning and synchronous map creation; the millimeter wave radar sensor is used for detecting the distance between the vehicle and the pedestrian and the distance between the vehicle and the traveling vehicle; the ultrasonic radar is used for detecting the distance between the short-distance vehicles; the vision sensor, the positioning system and the inertial navigation system are used for realizing the vehicle positioning technology.

Further, the different scenes in the step (2) include five scenes including a cross-sea bridge road condition in rainy and snowy weather, a cross-sea bridge road condition in strong wind and severe weather, a road condition when the bridge vibrates in sunny days, a running road condition of a bicycle in frequent and changeable weather, and a running road condition of a plurality of bicycles in frequent and changeable weather.

Further, the data set of step (3) includes vehicle speed, travel track, vehicle position, heading angle, slip angle, yaw rate, roll angle.

Further, the network design of the DDPG algorithm in the step (4) is as follows:

an actor network is constructed, the vehicle state and the environment state are taken as input, the output is a vector formed by steering angle, accelerator and brake signals, the vector corresponds to 3 neurons of an actor strategy network output layer respectively, the activation function of the accelerator and the brake is set to be Sigmoid, the activation function of the steering action value is Tanh, and the hidden layer has the structure that: the first layer is a convolution size 7*7, a filter size 48, a step size 4, and a total of 200 neurons; the second layer is a convolution size 5*5, a filter size 16, a step size 2, an activation function ReLu function, 400 neurons total; the third layer adds 100 neurons to the LSTM layer; the fourth layer is a 128-unit fully connected layer; the fifth layer is a fully connected layer, totaling 128 units; the input of the critic network is a state and action space, and the state and action space are spliced with an activation function ReLu through two hidden layers, namely 200 neurons in the first layer and 400 neurons in the second layer, so that a Q value is finally obtained; definition of h _i ∈(S _t-T ,S _t-T+1 ,…,S _t ) Wherein S is _t-T And S is _t State information respectively representing the current time and the current time, the encoded state is: s=f (h _i The method comprises the steps of carrying out a first treatment on the surface of the β), then the policy of the changed actor network is defined as: a=μ (h _i /β,γ ^π )+η。

Further, the implementation process of the step (5) for implementing the tracking control of the path track under the normal running road condition of the automatic driving vehicle is as follows:

the method comprises the steps of establishing a vehicle dynamics model by considering roll, sideslip and yaw dynamics characteristics of a vehicle under normal running road conditions, namely road conditions when a bridge vibrates on a sunny day, setting vehicle state constraint conditions, and determining a lateral stability range, a maximum steering angle range and a range of allowable vehicle control for preventing roll so as to reduce a lateral deviation error of the vehicle:

ω _z-min ≤ω _z ≤ω _z-max ,ω _x-min ≤ω _x ≤ω _x-max ，u _x-min ≤u _x ≤u _x-max ，e _r-x-min ≤e _r ≤e _r-x-max

wherein omega is _z Is yaw rate; omega _x Is the roll angle of the vehicle; u (u) _x Is the steering angle; e, e _r Is a lateral tracking offset error;

according to LSTM predicted road state information, constructing an objective function considering steering angle, tire attachment coefficient, roll angle error and path tracking error, and determining physical constraint of the vehicle determined by transverse tracking offset error under the condition of full consideration of the dynamics constraint condition of maximum allowable error of the vehicle so as to reduce the error of tracking control of the vehicle:

wherein w is ₁ ,w ₂ ,w ₃ ,w ₄ Parameter variables respectively; mu (mu) _r Is the road adhesion coefficient;

adopting an iterative control algorithm to realize compensation control of vehicle roll, and setting a reference vehicle state, a reference control input and a reference output value to ensure a tracking function under the conditions of vehicle physical constraint and road constraint under the condition of multiple constraints, thereby improving the anti-interference performance of the vehicle during running and reducing a model error rate;

according to the running condition of the vehicle, constructing a state space, an action space and a reward function required by a DDPG algorithm; the motion space mainly comprises steering wheel rotation angle, throttle and braking signals, and the state space comprises vehicle transverse tracking error and change rate thereof, vehicle side-tipping angle error and change rate thereof, and yaw rate error and change rate thereof; construction of the bonus function in case of vibration of the cross-sea bridge, the actual track of the vehicle is changed, the road is inclined at a certain angle, and thus the bonus function is equal to the cumulative multiplication of the discount factor and the change of speed.

Further, the implementation process of the step (5) for implementing the tracking control of the path track of the automatic driving vehicle under the extreme driving road condition is as follows:

under extreme driving road conditions, namely under severe weather and other factors affecting road conditions, the phenomenon that the vehicle easily generates wet skid and vibration is considered, the driving track is changed, the actual running speed of the vehicle is affected, and the actual vehicle speed and the planned vehicle speed are deviated, so that the change of the actual vehicle speed is set as follows:

in the formula, v _ref Is the reference vehicle speed; v _act Is the actual vehicle speed; alpha is an influencing factor;

when the error between the actual vehicle speed and the reference vehicle speed is within 2KM/H, the actual vehicle speed can be assumed to be equal to the reference vehicle speed; when the error between the actual vehicle speed and the reference vehicle speed is in the [2 ] KM/H interval, the vehicle speed of the actual vehicle speed is equal to the reference vehicle speed and the difference between the two vehicle speeds; wherein α is a speed change factor; when the error between the actual vehicle speed and the reference vehicle speed is greater than 5KM/H, the vehicle needs to be braked instantaneously at the moment, so that the running safety of the vehicle is ensured;

at time t, the vehicle's set of actions and states are expressed as follows: { v ₁ ,…,v _i …v _n I=1, …, n, path planning is implemented using bezier curves to produce collision-free predicted trajectories; in order to judge the accuracy of vehicle speed planning, the vehicle state and action of a small sample are taken out from an experience buffer of a DDPG algorithm as reference values: { v _r1 ,…,v _ri …v _rn I=1, …, n, and the error of both is calculated as:

in the formula, v _i Is the vehicle speed; l is the vehicle speed error rate; v _ri The reference vehicle speed is obtained from the experience buffer;

under the extreme driving condition, the expected state reference track is given as S _r The output error is e _k (t)＝S _r (t) -S (t), learning law: u (u) _k+1 (t)＝L(u _k (t),e _k (t)) to obtain the compensation control action a _k The method comprises the steps of carrying out a first treatment on the surface of the Under normal running, the DDPG algorithm is adopted to realize vehicle control, and the output action is a _π ＝μ(h _i /β,γ ^π ) +η, total action control a=a _k +a _π The reference formula is as follows:

in order to judge the accuracy of path planning, the track error when the actual running track of the actual running vehicle of the vehicle and the reference track generate the change of the side-tilt angle is calculated as follows:

wherein r is _act Is the actual vibrating vehicle track; r is (r) _ref Is a reference track;

the angle difference between the actual track and the reference track is the largest, and mu is the road adhesion coefficient; sigma is the deviation angle when the vehicle generates lateral sideslip, phi is the vehicle side-tipping angle, d is the vertical vibration distance, χ is the deviation angle when the bridge vibrates vertically, namely, the vertical dip angle generated by the bridge deck;

the vertical dip angle χ that bridge vibration produced can be maximally set to:

searching an optimal road running area according to a high-precision map designed by the visual sensor, designing a reference vehicle state, a control input value and a parameter output value in the optimal road running area, designing constraint conditions of multiple types of states, and realizing roll control of the vehicle by adopting an iterative learning control algorithm;

the prediction states of the high-precision map and the LSTM designed by the visual sensor find a limited road driving area, and design a range of layered uncertainty state parameters and action parameters, when a vehicle is driven in the limited road driving area, the DDPG and an iterative control algorithm are adopted to realize the roll control of the vehicle, the iterative control algorithm plays a role in compensation, and a reward function is constructed according to the constraint range of the vehicle state, wherein the reward function is as follows:

R＝v·(R ₁ +R ₂ +…R ₆ )

wherein R is ₁ And R is ₂ Each representing a lateral distance error and a rate of change thereof; r is R ₃ And R is ₄ Representing the transverse angular velocity and its rate of change; r is R ₅ And R is ₆ Representing the roll angle and its rate of change; x is an angle value; v is the vehicle speed; e, e _y Is the lateral distance, k _i ，k _j The bonus factors, respectively.

The beneficial effects are that: compared with the prior art, the invention has the beneficial effects that: 1. the invention designs a comprehensive control method for the roll of the automatic driving vehicle based on a reinforcement learning algorithm (DDPG), and the roll of the vehicle is controlled in a complex road environment through reinforcement learning, so that the automatic driving vehicle can realize exploratory running of the vehicle under complex road conditions and extreme weather through exploration and utilization methods; 2. and the compensation effect of iterative learning control on the DDPG algorithm is carried out aiming at extreme driving conditions, so that the comprehensive control effect of the vehicle is realized, and the final safe driving of the vehicle is ensured.

Drawings

Fig. 1 is a schematic diagram of a DDPG network architecture;

FIG. 2 is a schematic diagram of integrated control of vehicle roll;

fig. 3 is a flow chart of integrated control based on autonomous vehicle roll.

Detailed Description

The invention is described in further detail below with reference to the accompanying drawings.

The invention provides an automatic driving vehicle roll control method based on DDPG and iterative control, which specifically comprises the following steps:

step 1: laser radar, vision, millimeter wave radar, ultrasonic radar sensor, positioning system and inertial navigation system are installed on an autonomous vehicle.

The invention is oriented to the road condition environment of a cross-sea bridge, and aims to control the vehicle to safely run at a medium-low speed (5-80 KM/H), and realize high-level intellectualization of the vehicle through the control behavior of the vehicle with instantaneous side inclination. To achieve the above object, the present invention mounts a plurality of several laser radar, machine vision, millimeter wave radar and ultrasonic radar sensors on an autonomous vehicle, and mounts a positioning system, an inertial navigation system (IMU), and the like. The laser radar sensor is used for detecting dynamic and static obstacles on the road, including pedestrians, motorcycles, various vehicles and the like, and can drive a road area; the machine vision sensor is used for sensing lane lines, pedestrians and vehicles and performing positioning and synchronous map creation; the millimeter wave radar sensor is used for detecting the distance between the vehicle and the pedestrian and the distance between the vehicle and the traveling vehicle; the ultrasonic radar is used for detecting the distance between the short-distance vehicles; positioning systems and inertial navigation Systems (IMUs) are used to implement vehicle positioning technology.

Step 2: the vision sensor, the positioning system and the inertial navigation system are used for realizing the positions and the maps of the vehicle in different scenes respectively so as to generate the running maps of the automatic driving vehicle in different scenes and realize the environment required by the running track of the vehicle.

The method comprises the steps of using a visual sensor, a positioning system and an inertial navigation system (IMU) to realize positions and maps of vehicles in severe weather such as sunny days, rainy days, snowy days, foggy days, strong winds and the like respectively so as to generate an automatic driving vehicle driving map in five scenes of a cross-sea bridge road condition in rainy and snowy days, a cross-sea bridge road condition in strong wind and severe weather, a road condition when a bridge vibrates in sunny days, a driving road condition of a bicycle in frequent changeable weather and a driving road condition of a plurality of vehicles in frequent changeable weather, and the method is used for realizing an environment required by a vehicle driving track.

Step 3: and respectively controlling a steering wheel, an accelerator and a pedal, driving on a cross-sea bridge, acquiring corresponding driving tracks in rainy and snowy days, strong wind and severe weather and sunny days, and constructing a data set.

A driver with abundant experience respectively runs on the cross-sea bridge under five scenes by controlling the steering wheel, the accelerator and the pedal, and records corresponding running tracks so as to construct corresponding data sets. The data set includes: the vehicle speed, the running track, the vehicle position, the course angle, the slip angle, the yaw rate and the roll angle provide necessary reference data for training the data set and evaluating the controllability of the vehicle.

Step 4: the DDPG algorithm is trained on the running map of the automatic driving vehicle under different scenes and is used for running states of the cross-sea bridge under different complex road condition grades in severe weather; the automatic driving vehicle generates real-time vehicle states through interaction with map environments in different scenes, and determines action behaviors of the vehicle; when motion training is performed, initializing a motion space, generating state space information by an online strategy network in an actor network, performing motion output, and adding motion noise to acquire a exploratory motion space.

As shown in fig. 1, an actor network is constructed, the vehicle state and the environment state are taken as input, the output is a vector formed by steering angle, accelerator and brake, 3 neurons of an actor strategy network output layer are respectively corresponding to the actor, the activation function of the accelerator and the brake is set to be Sigmoid, the activation function of the steering action value is Tanh, and the structure of the hidden layer is as follows: the first layer is a convolution size 7*7, a filter size 48, a step size 4, and a total of 200 neurons; the second layer is a convolution size 5*5, a filter size 16, a step size 2, an activation function ReLu function, 400 neurons total; the third layer adds 100 neurons to the LSTM layer; the fourth layer is a 128-unit fully connected layer; the fifth layer is a fully connected layer, totaling 128 units; the input of the critic network is a state and action space, and the state and action space are spliced with an activation function ReLu through two hidden layers, namely 200 neurons in the first layer and 400 neurons in the second layer, so that a Q value is finally obtained; definition of h _i ∈(S _t-T ,S _t-T+1 ,…,S _t ) Wherein S is _t-T And S is _t State information respectively representing the current time and the current time, the encoded state is: s=f (h _i The method comprises the steps of carrying out a first treatment on the surface of the β), then the policy of the changed actor network is defined as: a=μ (h _i /β,γ ^π )+η。

Designing an action space of the vehicle, wherein the action space comprises a steering wheel angle delta and a braking signal of the vehicle

And throttle signal

Considering that the running environment of the vehicle is complex, when the road environment is complex, the vehicle is in variable speed running, and the braking signal is the action generated by the vehicle under the extreme running condition so as to prevent the vehicle from rolling and rollover movement caused by braking and road surface wet sliding, and the action space at the moment should be set into three types of steering wheel rotation angle, braking signal and accelerator signal; when the vehicle is under the normal running working condition, the vehicle is assumed to run at a constant speed, in order to prevent the vehicle from generating a roll phenomenon due to wet and slippery road surface, the action space is set to be a steering wheel angle and an accelerator signal, and the constraint ranges of three actions are set according to the two different running working conditions, so that the vehicle can be ensured to run in a controllable way in a running possible road area.

The vehicle is configured to obtain state data by exploring and utilizing the environment, and the generated data states generally comprise a lateral distance and a change rate thereof, and a vehicle roll angle and a change rate thereof, and the state data are generally contained in an experience buffer. When the iterative learning is adopted to realize the vehicle control, parameters such as the state of a reference vehicle, the track of the vehicle and the like need to be designed, the reference state and the track can be obtained from an experience buffer, and the reference track can be adjusted and changed according to the complexity of different road environments.

Under five different scenes of road conditions of a cross-sea bridge in rainy and snowy weather, road conditions of a cross-sea bridge in severe weather, road conditions of a bridge in vibration in sunny weather, running road conditions of a bicycle in frequent variable weather and running road conditions of a plurality of bicycles in frequent variable weather, different generated tracks are used as reference paths, error comparison is carried out between an actually planned path of a vehicle and the reference tracks, and the reference tracks also need to be added with various constraint conditions meeting vehicle dynamics characteristics to modify and adjust the constraint conditions as set actual paths, so that the method can be expressed as follows:

where σ is a path influencing factor, p _ref Is a reference track; p is p _act An actual trajectory; that is, when the vehicle travels in different road environments, the acquired travel track needs to be appropriately modified and adjusted to conform to the vehicle dynamics characteristics of the vehicle during automatic traveling, and the travel track can be used as the travel track of the automatic driving vehicle.

Step 5: as shown in fig. 2, based on LSTM history memory and road planning attributes, a predicted path of the state of the autonomous vehicle is generated, tracking control of the path trajectories of the autonomous vehicle under normal driving road conditions and extreme driving road conditions is realized by using a DDPG algorithm, and compensation control of the autonomous vehicle is realized by using an iterative control method.

In order to ensure the safety and stability of the vehicle under normal driving road conditions, namely road conditions when the bridge vibrates on sunny days in five scenes, the roll, sideslip and yaw dynamics characteristics of the vehicle are required to be considered in vehicle dynamics modeling, vehicle state constraint conditions are set, and a lateral stability range, a maximum steering angle range and a range for allowable vehicle control for preventing roll are determined so as to reduce the lateral deviation error of the vehicle:

wherein ψ= [ v _y ω _z ω _x φ] ^T Is a state vector, u ₁ Is a control input, u ₂ Is an auxiliary control input; omega _z Is yaw rate; omega _x Is the roll angle of the vehicle; u (u) _x Is the steering angle; e, e _r Is a lateral tracking offset error.

wherein w is ₁ ,w ₂ ,w ₃ ,w ₄ Parameter variables respectively; mu (mu) _r Is the road adhesion coefficient

As shown in fig. 3, the iterative control algorithm is adopted to realize the compensation control of the vehicle roll, and the reference vehicle state, the reference control input and the reference output value are set to ensure the tracking function under the conditions of the physical constraint of the vehicle and the road constraint under the multiple constraint conditions, thereby increasing the anti-interference performance of the vehicle during running and reducing the model error rate. Firstly, a DDPG algorithm is adopted to carry out training work of a network model, interactive training is carried out through road conditions of a vehicle and a dynamic cross-sea bridge, a training completion task is ensured, if the task is completed, a trained action is saved, if the training task is not ideal in completion effect, an iterative control algorithm is adopted to compensate parameters of an output action space, finally, the training task is completed, a better action is realized, and finally, a better automatic driving vehicle roll control is realized.

Under extreme driving conditions, namely, the road conditions of cross-sea bridges in rainy and snowy weather in five scenes, the road conditions of cross-sea bridges in strong wind and severe weather, the driving road conditions of single vehicles in frequent and varied weather and the driving road conditions of multiple vehicles in frequent and varied weather; the road environment has uncertainty due to bad weather influence, which interferes with the normal running of the vehicle.

Under extreme driving road conditions, the vehicle is easy to generate wet skid and vibration phenomena, the driving track can be changed to influence the actual running speed of the vehicle, and the actual vehicle speed is deviated from the planned vehicle speed, so that the change of the actual vehicle speed can be set as follows:

in the formula, v _ref Is the reference vehicle speed; v _act Is the actual vehicle speed; alpha is an influencing factor.

When the error between the actual vehicle speed and the reference vehicle speed is within 2KM/H, the actual vehicle speed can be assumed to be equal to the reference vehicle speed; when the error between the actual vehicle speed and the reference vehicle speed is in the [2 ] KM/H interval, the vehicle speed of the actual vehicle speed is equal to the reference vehicle speed and the difference between the two vehicle speeds, wherein alpha is a speed change factor; when the error between the actual vehicle speed and the reference vehicle speed is larger than 5KM/H, the vehicle needs to be braked instantaneously, and the running safety of the vehicle is ensured.

After the path prediction of the automatic driving vehicle passes through the LSTM historic memory state, the predicted speed path is generated by utilizing the road planning attribute, and at the time t, the action and state set of the vehicle are expressed as follows: { v ₁ ,…,v _i …v _n I=1, …, n, path planning is implemented using bezier curves to produce collision-free predicted trajectories; in order to judge the accuracy of vehicle speed planning, the vehicle state and action of a small sample are taken out from an experience buffer of a DDPG algorithm as reference values: { v _r1 ,…,v _ri …v _rn I=1, …, n, and the error of both is calculated as:

in the formula, v _i Is the vehicle speed; l is the vehicle speed error rate; v _ri Is the reference vehicle speed obtained from the experience buffer.

Under extreme conditions, given a desired state reference trajectory of S _r The output error is e _k (t)＝S _r (t) -S (t), learning law: u (u) _k+1 (t)＝L(u _k (t),e _k (t)) to obtain the compensation control action a _k The method comprises the steps of carrying out a first treatment on the surface of the Under normal running, the DDPG algorithm is adopted to realize vehicle control, and the output action is a _π ＝μ(h _i /β,γ ^π ) +η, total action control a=a _k +a _π The reference formula is as follows:

in order to judge the accuracy of path planning, the track error when the actual running track of the actual running vehicle of the vehicle and the reference track generate the change of the side-tilt angle is calculated as follows: the rolling motion phenomenon of the vehicle is mainly shown in two cases, wherein the first motion is that when a bridge vibrates, the vehicle automatically drives to deviate from a planned path, and at the moment, the vehicle easily rolls to generate a roll angle; therefore, a track error in which the actual running track of the vehicle and the reference track produce a change in the roll angle can be expressed as:

the angle difference between the actual track and the reference track is the largest, sigma is the deviation angle when the vehicle generates lateral sideslip, phi is the vehicle side-tipping angle, d vertical vibration distance, χ is the deviation angle when the bridge vibrates vertically, namely, the vertical dip angle generated by the bridge deck.

the rolling motion phenomenon of the vehicle occurs, the second motion is that the road surface is wet and slippery due to bad weather, the road attachment coefficient is changed, and the rolling, sideslip and side turning motions of the vehicle are caused; therefore, a track error in which the actual running track of the vehicle and the reference track generate a roll angle change can be expressed as:

the angle difference between the actual track and the reference track is the largest, and mu is the road adhesion coefficient; sigma is the angle of departure when the vehicle is laterally sideslip, phi is the roll angle of the vehicle, d is the vertical vibration distance, χ is the angle of departure when the bridge is vertically vibrating, i.e., the vertical tilt angle produced by the deck, μ is the road attachment coefficient, which is in the range of [01 ]]。

According to a high-precision map designed by the visual sensor, an optimal road running area is searched, and in the optimal road running area, the reference vehicle state, the control input and the parameter output values are designed, meanwhile, constraint conditions of multiple types of states are designed, and the roll control of the vehicle is realized by adopting an iterative learning control algorithm, so that the DDPG algorithm plays a compensation role, and the safe running of the vehicle in the controllable road area is realized. Searching a limited road running area according to a high-precision map designed by a visual sensor and the prediction state of an LSTM, designing the range of a layered uncertainty state parameter and an action parameter, and realizing the roll control of the vehicle by adopting a DDPG and an iterative control algorithm when the vehicle runs in the limited road running area, wherein the iterative control algorithm plays a role in compensation; when the vehicle is in a limit working condition, constraint conditions of the vehicle state need to be added, and a reward function is constructed as follows:

R＝v·(R ₁ +R ₂ +…R ₆ )

wherein R is ₁ And R is ₂ Each representing a lateral distance error and a rate of change thereof; r is R ₃ And R is ₄ Representing the transverse angular velocity and its rate of change; x is an angle value; v is the vehicle speed; r is R ₅ And R is ₆ Representing the roll angle and its rate of change; e, e _y Is the lateral distance, k _i ，k _j The bonus factors, respectively.

The above description is only specific for the practical embodiments of the present invention, and they are not intended to limit the scope of the invention, but all equivalent manners or modifications that do not depart from the technology of the present invention should be included in the scope of the invention.

Claims

1. An automatic driving vehicle roll control method based on DDPG and iterative control, which is characterized by comprising the following steps:

(5) Based on LSTM historic memory and road planning attributes, generating a predicted path of the state of the automatic driving vehicle, adopting a DDPG algorithm to realize tracking control of the path track of the automatic driving vehicle under normal driving road conditions and extreme driving road conditions, and adopting an iterative control method to realize compensation control of the automatic driving vehicle;

the network design of the DDPG algorithm in the step (4) is as follows:

an actor network is constructed, the vehicle state and the environment state are taken as input, the output is a vector formed by steering angle, accelerator and brake signals, the vector corresponds to 3 neurons of an actor strategy network output layer respectively, the activation function of the accelerator and the brake is set to be Sigmoid, the activation function of the steering action value is Tanh, and the hidden layer has the structure that: the first layer is a convolution size 7*7, a filter size 48, a step size 4, and a total of 200 neurons; the second layer is a convolution size 5*5, a filter size 16, a step size 2, an activation function ReLu function, 400 neurons total; the third layer adds 100 neurons to the LSTM layer; the fourth layer is a 128-unit fully connected layer; the fifth layer is a fully connected layer, totaling 128 units; the input of the critic network is a state and action space, and the state and action space are spliced with an activation function ReLu through two hidden layers, namely 200 neurons in the first layer and 400 neurons in the second layer, so that a Q value is finally obtained; definition of h _i ∈(S _t-T ,S _t-T+1 ,…,S _t ) Wherein S is _t-T And S is _t State information respectively representing the current time and the current time, the encoded state is: s=f (h _i The method comprises the steps of carrying out a first treatment on the surface of the β), then the policy of the changed actor network is defined as action a=μ (h _i /β,γ ^π )+η；

The implementation process of the tracking control of the path track of the automatic driving vehicle under the normal driving road condition in the step (5) is as follows:

2. The DDPG and iterative control-based roll control method for an autonomous vehicle of claim 1, wherein the lidar sensor of step (1) is used to detect dynamic and static obstacles on roads, including pedestrians, motorcycles, and various vehicles, and movable road areas; the visual sensor is used for sensing lane lines, pedestrians and vehicles and performing positioning and synchronous map creation; the millimeter wave radar sensor is used for detecting the distance between the vehicle and the pedestrian and the distance between the vehicle and the traveling vehicle; the ultrasonic radar is used for detecting the distance between the short-distance vehicles; the vision sensor, the positioning system and the inertial navigation system are used for realizing the vehicle positioning technology.

3. The DDPG and iterative control-based automatic driving vehicle roll control method of claim 1, wherein the different scenes in the step (2) comprise five scenes including a cross-sea bridge road condition in rainy and snowy weather, a cross-sea bridge road condition in strong wind and severe weather, a road condition when a bridge vibrates on a sunny day, a driving road condition of a single vehicle in frequent variable weather, and a driving road condition of multiple vehicles in frequent variable weather.

4. The DDPG and iterative control-based automatic driving vehicle roll control method of claim 1, wherein the dataset of step (3) comprises vehicle speed, travel trajectory, vehicle position, heading angle, slip angle, yaw rate, roll angle.

5. The DDPG and iterative control-based roll control method for an autonomous vehicle according to claim 1, wherein the implementation of the step (5) for implementing the tracking control of the path trajectory of the autonomous vehicle under the extreme driving road condition is as follows:

when the error between the actual vehicle speed and the reference vehicle speed is within 2KM/H, the actual vehicle speed is equal to the reference vehicle speed; when the error between the actual vehicle speed and the reference vehicle speed is in the [2 ] KM/H interval, the vehicle speed of the actual vehicle speed is equal to the reference vehicle speed and the difference between the two vehicle speeds; wherein α is a speed change factor; when the error between the actual vehicle speed and the reference vehicle speed is greater than 5KM/H, the vehicle needs to be braked instantaneously at the moment, so that the running safety of the vehicle is ensured;

under the extreme driving condition, the expected state reference track is given as S _r The output error is e _k (t)＝S _r (t) -S (t), learning law:u _k+1 (t)＝L(u _k (t),e _k (t)) to obtain the compensation control action a _k The method comprises the steps of carrying out a first treatment on the surface of the Under normal running, the DDPG algorithm is adopted to realize vehicle control, and the output action is a _π ＝μ(h _i /β,γ ^π ) +η, total action control a=a _k +a _π The reference formula is as follows:

in order to judge the accuracy of path planning, the track error when the actual running track and the reference track of the vehicle generate the change of the roll angle is calculated as follows:

vertical inclination χ maximize that bridge vibration produced sets up to:

R＝v·(R ₁ +R ₂ +…R ₆ )