CN112896161A - Electric automobile ecological self-adaptation cruise control system based on reinforcement learning - Google Patents

Electric automobile ecological self-adaptation cruise control system based on reinforcement learning Download PDF

Info

Publication number
CN112896161A
CN112896161A CN202110171999.6A CN202110171999A CN112896161A CN 112896161 A CN112896161 A CN 112896161A CN 202110171999 A CN202110171999 A CN 202110171999A CN 112896161 A CN112896161 A CN 112896161A
Authority
CN
China
Prior art keywords
vehicle
module
electric automobile
follows
electric
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110171999.6A
Other languages
Chinese (zh)
Other versions
CN112896161B (en
Inventor
翟春杰
杨建�
杨祥宇
颜成钢
孙垚棋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202110171999.6A priority Critical patent/CN112896161B/en
Publication of CN112896161A publication Critical patent/CN112896161A/en
Application granted granted Critical
Publication of CN112896161B publication Critical patent/CN112896161B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/14—Adaptive cruise control
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/10—Longitudinal speed
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/40—Dynamic objects, e.g. animals, windblown objects
    • B60W2554/404—Characteristics
    • B60W2554/4042—Longitudinal speed
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/80—Spatial relation or speed relative to objects
    • B60W2554/802—Longitudinal distance

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Electric Propulsion And Braking For Vehicles (AREA)

Abstract

The invention discloses an electric automobile ecological self-adaptive cruise control system based on reinforcement learning, which comprises an information acquisition module, a longitudinal dynamics module, an electric automobile energy storage module, a control target module and a controller design module, wherein the information acquisition module is used for acquiring information of an electric automobile; the information acquisition module acquires position and speed information of a front vehicle through a radar and a vehicle-mounted information sensor; the longitudinal dynamics module is used for calculating acceleration, lumped resistance, actual vehicle distance, wheel torque and expected power; the electric vehicle energy storage module is used for calculating required power and resistance under the driving and braking conditions of the electric vehicle; the control target module ensures the safety of the vehicle by restricting the distance between the vehicles; the energy-saving driving is improved and the service life of a battery is prolonged by setting an optimization target; the controller design module is used for determining the state variables and the specific involved contents of the cost function in the control process. The system can ensure the following performance of the electric automobile, realize the driving safety, enhance the energy economy and prolong the service life of the battery.

Description

Electric automobile ecological self-adaptation cruise control system based on reinforcement learning
Technical Field
The invention belongs to automobile auxiliary intelligent driving, and particularly relates to an electric automobile ecological self-adaptive cruise control system based on self-adaptive dynamic programming.
Background
Currently, the automobile industry faces a greater pressure of energy conservation and emission reduction as a large industrial user with greater energy consumption. Zero emission of pollutants in the driving process of the electric automobile is an important direction for future development of the automobile industry. How to apply the intelligent driving technology to the electric automobile to further develop the energy-saving potential of the electric automobile is a key research direction of various colleges and universities and vehicle enterprises. An Advanced Driver Assistance System (ADAS) is an initial development stage of an intelligent driving technology, and can automatically acquire relevant environmental data by using various vehicle-mounted sensors, realize automatic control on a vehicle, and improve driving comfort and active safety.
As an advanced intelligent driving assistance system, Adaptive Cruise Control (ACC) is developed from early Cruise Control, and is mainly used to Control longitudinal movement of a vehicle. The ACC system can use various vehicle-mounted sensors to detect the relative position and speed of a vehicle in front, and automatically adjust the speed of the vehicle according to a control strategy so as to keep an expected safe distance, thereby being beneficial to improving the traffic flow, reducing traffic accidents and providing comfortable driving experience.
Although the ACC system can maintain a certain safe vehicle distance and reduce energy consumption by reducing air resistance, its energy saving effect is not significant, especially for passenger vehicles with small vehicle head area. Particularly, when the electric automobile is controlled by the ACC system based on a constant vehicle distance and a constant time distance, the electric automobile often follows the front vehicle closely at a certain vehicle distance, and if the speed fluctuation of the front vehicle is large, the electric automobile is always in a frequent acceleration and deceleration state, which greatly affects the service life of a battery in the electric automobile and also causes energy consumption loss and driving discomfort.
Executing an Action-Dependent Heuristic Dynamic Programming (ADHDP) framework, that is, referring to book "intelligent optimization control based on adaptive Dynamic Programming" 4.3 ADHDP algorithm based on BP network and implementing P118, author: linxiaofeng, Sonshaoshangjian, Songchuning.
Disclosure of Invention
The invention aims to provide an enhanced Learning (RL) -based Eco-Adaptive Cruise Control (Eco-ACC) system for an electric vehicle, which can ensure the following performance of the electric vehicle, realize driving safety, enhance energy economy and prolong the service life of a battery.
An electric automobile ecological self-adaptive cruise control system based on reinforcement learning comprises an information acquisition module, a longitudinal dynamics module, an electric automobile energy storage module, a control target module and a controller design module.
The information acquisition module acquires position and speed information of a front vehicle through a radar and a vehicle-mounted information sensor;
the longitudinal dynamics module is used for calculating acceleration, lumped resistance, actual vehicle distance, wheel torque and expected power;
the electric vehicle energy storage module is used for calculating required power and resistance under the driving and braking conditions of the electric vehicle;
the control target module ensures the safety of the vehicle by restricting the distance between the vehicles; the energy-saving driving is improved and the service life of a battery is prolonged by setting an optimization target;
the controller design module is used for determining the state variables and the specific involved contents of the cost function in the control process.
An enhanced learning-based ecological adaptive cruise control method for an electric vehicle adopts an execution-Dependent Heuristic Dynamic Programming (ADHDP) framework, and comprises the following steps of:
1) the state variable x (t) is determined through an information acquisition module and a controller design module, the utility function U (t) is determined through a control target module, and relevant parameters are initialized. (ii) a
2) Inputting the state variable x (t) into the execution network acquisition control variable u (t);
3) inputting the state variable x (t) and the control variable u (t) into an evaluation network to obtain an expected cost J (t);
4) setting errors of an execution network and an evaluation network;
5) and solving a state variable x (t +1) at the next moment through the longitudinal dynamics module and the electric automobile energy storage module.
6) Updating the weight value of the execution network, and inputting a state variable x (t +1) into the execution network to obtain a control variable u (t + 1);
7) updating the weight of the evaluation network, and obtaining the expected cost through the evaluation network
Figure BDA0002939020610000021
A value;
8) and judging whether the evaluation network and the execution network meet the maximum iteration times or whether the tolerance meets the self-adaptive iteration value. If the control variable u (t +1) is satisfied, the solved control variable u (t +1) is used as the optimal or suboptimal control variable, otherwise, the second step is returned.
The invention has the following beneficial effects:
the speed of the vehicle controlled by the system is basically consistent with that of the vehicle in front, and the acceleration of the vehicle is smoother than that controlled by the traditional ACC system, so that passengers feel more comfortable; the actual distance between the vehicle controlled by the system and the vehicle in front is always kept in a safe range, so that the safety of the vehicle in the driving process is ensured; the vehicle controlled by the system of the invention is more energy-saving than the vehicle controlled by the traditional ACC system. The system can ensure the following performance of the electric automobile, realize the driving safety, enhance the energy economy and prolong the service life of the battery.
Drawings
FIG. 1 is a vehicle following scenario;
FIG. 2 is a block diagram of the ADHDP architecture;
FIG. 3 is an ADHDP evaluation network structure;
FIG. 4 is an ADHDP execution network structure;
FIG. 5 is a flow chart based on the ADHDP control algorithm;
FIG. 6 is a comparison of UDDS driving cycle simulation results;
FIG. 7 is a comparison of results of MANHATAN driving cycle simulation;
fig. 8 is a comparison of WLTC2 driving cycle simulation results.
Detailed Description
The system and method of the present invention are further described with reference to the accompanying drawings and examples.
An electric automobile ecological self-adaptive cruise control system based on reinforcement learning comprises an information acquisition module, a longitudinal dynamics module, an electric automobile energy storage module, a control target module and a controller design module.
The information acquisition module acquires position and speed information of a front vehicle through a radar and a vehicle-mounted information sensor;
the longitudinal dynamics module is used for calculating acceleration, lumped resistance, actual vehicle distance, wheel torque and expected power;
the electric vehicle energy storage module is used for calculating required power and resistance under the driving and braking conditions of the electric vehicle;
the control target module ensures the safety of the vehicle by restricting the distance between the vehicles; the energy-saving driving is improved and the service life of a battery is prolonged by setting an optimization target;
the controller design module is used for determining the state variables and the specific involved contents of the cost function in the control process.
A vehicle following scene to be researched is shown in fig. 1, wherein a controlled electric vehicle and a vehicle in front of the controlled electric vehicle are respectively marked as a main vehicle and a front vehicle; the actual vehicle distance between the main vehicle and the front vehicle is represented by L; the speeds of the main and front vehicles being V respectivelyhAnd VpAnd (4) showing. The specific contents of each module are as follows:
a longitudinal dynamics module:
the longitudinal dynamics model of the master is represented as follows:
Figure BDA0002939020610000041
in the formula: sh(t)、vh(T) and Tw(t) position, velocity and wheel torque of the host vehicle, respectively; m, R, etatAnd delta is the main vehicle mass, the effective rolling radius of the tire, the transmission efficiency and the rotation inertia coefficient respectively; fb(t) and Fr(t) are each independentlyPower and collective resistance.
Lumped resistance F consisting of aerodynamic resistance, rolling resistance and gravityr(t)Fr(t) is represented as follows:
Figure BDA0002939020610000042
in the formula: phih(L(t))、Cd、μh、AvAnd theta(s)h(t)) respectively are a vehicle normalized resistance coefficient, an air resistance coefficient, a rolling resistance coefficient, a head windward area and a road surface gradient; g and ρ are the acceleration of gravity and the air density, respectively. Further, the distance L (t) between the host vehicle and the preceding vehicle can be expressed as
L(t)＝sp(t)-sh(t)-dcar (3)
In the formula: dcarIndicating the length of the main vehicle body, sp(t) represents a preceding vehicle position.
The torque of the wheel is output or input to the motor through the gear, and the torque T of the motormAnd a rotational speed omegamIs represented as follows:
Figure BDA0002939020610000043
in the formula: grThe fixed gear ratio of the main vehicle. Wheel speed omegawThe calculation formula of (t) is as follows:
Figure BDA0002939020610000051
then, the input power of the motor inverter is given as follows:
Figure BDA0002939020610000052
in the formula: etam(t)(0＜ηm(t) < 1) represents the efficiency of the motor inverter.
Electric automobile energy storage system module:
the variable symbols are defined as follows:
·Pbat(t): the output power of the battery pack at the time t;
·Pe(t): the required power of the electric automobile at the moment t;
·Vbat(t): open circuit voltage of the battery pack at time t;
·Ibat(t): the current of the battery pack at the time t;
·SoCbat(t): the State of Charge (SOC) of the battery pack at time t;
·Rbat，disch(SoCbat(t)): the discharge resistance of the battery pack at the time t;
·Rbat，ch(SoCbat(t)): the charging resistance of the battery pack at the time t;
discharge resistance R of battery packbat，disch(SoCbat(t)) and a charging resistor
Rbat，ch(SoCbat(t)) is represented as follows:
Figure BDA0002939020610000053
(1) a driving mode:
Figure BDA0002939020610000054
(2) regenerative braking mode:
Figure BDA0002939020610000055
the state of charge SoC of the battery is as follows:
Figure BDA0002939020610000056
a control target module:
(1) vehicle safety:
to ensure vehicle safety, constraints on the vehicle spacing are given as follows:
dmin(υh(t))≤L(t)≤dmax(υh(t)) (11)
wherein d ismin(υh(t)) and dmax(υh(t)) are respectively the minimum and maximum safe vehicle distances allowed, and their calculation formula is as follows:
Figure BDA0002939020610000061
(2) energy-saving driving:
in order to ensure the energy consumption economy of the vehicle during driving, the following optimization goals are given:
Figure BDA0002939020610000062
(3) and the service life of the battery is prolonged:
in order to reduce the battery capacity loss of the vehicle during running, the following optimization objectives are given:
Figure BDA0002939020610000063
a controller design module:
(1) bandstop function with compensation factor:
in order to obtain the error Δ d (t) between vehicles, the error of the iteration δ d (t) of the vehicle in the safety range is firstly obtained, which is specifically described as follows:
Figure BDA0002939020610000064
from the equation, Δ d (t) the inter-vehicle error can be found as:
Figure BDA0002939020610000065
wherein alpha is more than 0 and beta is more than or equal to 1
Figure BDA0002939020610000066
dmin，dmax∈R+Respectively the lower and upper band stop limits, cfIs a compensation factor.
In an optimization problem that minimizes the objective with a multi-objective cost function, the cost of the optimization is reduced when the parameters a, ss,
Figure BDA0002939020610000067
and cfAfter setting correctly, if the band stop function
Figure BDA0002939020610000068
As part of the cost function, the actual vehicle distance L (t) is limited to [ d [ [ d ]min，dmax]Within the range.
(2) Demand power optimization problem based on reinforcement learning:
first, the basic variables are defined:
x (t): state variables of the electric automobile at the moment t;
·Fb(t): braking force of the electric automobile at the time t;
·ωw(t): the wheel rotating speed of the electric automobile at the time t;
·ωm(t): the motor rotating speed of the electric automobile at the moment t;
·Tm(t): the motor torque of the electric automobile at the time t;
·Tm，max(t): the maximum motor torque allowed by the electric automobile at the time t;
u (t): control input of the electric automobile at the time t;
·ηm(t): the motor efficiency of the electric automobile at the moment t;
·Pe(t): the required power of the electric automobile at the moment t;
the continuous dynamic state equation of the host at time t is as follows:
Figure BDA0002939020610000071
in the formula: x (t) ═ Δ vh(t)，Δd(t)]TState variables representing the main vehicle dynamics system. After two types of variables are defined, the objective cost function J in the optimization problem is as follows:
Figure BDA0002939020610000072
in the formula: u is the utility function, γ is the reduction coefficient, 0 < γ ≦ 1, and J is the cost function for state x (t), which depends on the initial time t and the initial state x (t). The goal of reinforcement learning is to select a control sequence u (t) that minimizes the cost function defined by equation (18). In addition, the optimization goal of the objective cost function is as follows:
Figure BDA0002939020610000073
U(t)＝λ1L1+λ2L2+λ3L3. (20)
in the formula: considering the driving safety of the vehicle, L1With the aim of keeping the distance between the vehicles at a minimum distance dminAnd the maximum vehicle distance dmaxIn the meantime. In addition, the concentration of the alpha, beta,
Figure BDA0002939020610000074
and cfIs a parameter of the spacing stop band function. L is2The energy consumption economy of the vehicle during running can be improved. L is3The service life of the battery of the electric automobile can be prolonged.
Assuming the expected motor torque as the control variable, the control variables optimized based on the ADHDP algorithm are given as follows
u*(·|t0)＝argminJ(x(·|t0)) (21)
A flow chart of the overall control algorithm, as shown in FIG. 5;
the specific operation flow of the present invention is shown in fig. 3, and first we determine the state variable x (t) ═ Δ v, BSF by the controller module. Learning is then performed through our ADHDP framework to obtain the control best variables. The learning of the ADHDP learning framework comprises the following steps:
an enhanced learning-based ecological adaptive cruise control method for an electric vehicle adopts an execution-Dependent Heuristic Dynamic Programming (ADHDP) framework, as shown in fig. 2, and the structures of an evaluation network and an execution network based on a BP neural network are shown in fig. 3 and 4, and the method comprises the following steps:
1) the state variable x (t) is determined through an information acquisition module and a controller design module, the utility function U (t) is determined through a control target module, and relevant parameters are initialized.
2) Inputting state variable x (t) into execution network acquisition control variable u (t)
3) Inputting the state variable x (t) and the control variable u (t) into an evaluation network to obtain the expected cost
Figure BDA0002939020610000081
4) Setting errors of execution network and evaluation network
5) And solving a state variable x (t +1) at the next moment through the longitudinal dynamics module and the electric automobile energy storage module.
6) Updating the weight of the execution network, inputting the state variable x (t +1) to the execution network to obtain the control variable u (t +1)
7) Updating the weight of the evaluation network, and obtaining the expected cost through the evaluation network
Figure BDA0002939020610000082
Value of
8) And judging whether the evaluation network and the execution network meet the maximum iteration times or whether the tolerance meets the self-adaptive iteration value. If the control variable u (t +1) is satisfied, the solved control variable u (t +1) is used as the optimal or suboptimal control variable, otherwise, the second step is returned.
At the beginning of the learning process, the parameters of the evaluation network and the execution network are initialized randomly. In each time step after the simulation begins, iteration is carried out on the weight of the evaluation network until the maximum iteration number N is reachedcOr EcTo an allowable tolerance TcAfter iteration is terminated, obtaining an approximate value function from the evaluation network; iterating the weights of the execution network until a maximum number of iterations N is reachedahOr EaTo an allowable tolerance TcAfter iteration is terminated, obtaining control input from the execution network, and obtaining the optimal required power P through calculationeAnd applied to the host vehicle. The simulation parameters are shown in table 1.
TABLE 1 Online learning parameters
Figure BDA0002939020610000091
The inventive Eco-ACC control system was evaluated and tested using driving cycles such as urban, high speed, suburban, etc. The leading vehicle runs along the speed track of the driving cycle, and the following vehicle respectively adopts the traditional ACC system and the inventive Eco-ACC system to follow the leading vehicle. The test data for the typical UDDS, MANHATTAN and WLTC2 driving cycles are shown in fig. 6, 7 and 8, respectively, it being noted that to facilitate viewing of the simulation results, only the simulation effect plot for the first 400 seconds is shown in the simulation plot; the test data for more driving cycles are shown in tables 2 and 3.
TABLE 2 loss of Battery Capacity (%)
Figure BDA0002939020610000092
TABLE 3 loss of energy consumption (w.h)
Figure BDA0002939020610000093
Figure BDA0002939020610000101
Simulation results show that: the speeds of the vehicle controlled by the Eco-ACC system are basically consistent with those of the vehicle in front, and the acceleration of the vehicle is smoother than that controlled by the traditional ACC system, so that passengers feel more comfortable; the actual distance between the vehicle controlled by the Eco-ACC system and the vehicle in front is always kept in a safe range, so that the safety of the vehicle in the driving process is ensured; the Eco-ACC system controlled vehicle is more energy efficient than the conventional ACC system controlled vehicle, see table 3.

Claims (6)

1. An electric automobile ecological self-adaptive cruise control system based on reinforcement learning is characterized by comprising an information acquisition module, a longitudinal dynamics module, an electric automobile energy storage module, a control target module and a controller design module;
the information acquisition module acquires position and speed information of a front vehicle through a radar and a vehicle-mounted information sensor;
the longitudinal dynamics module is used for calculating acceleration, lumped resistance, actual vehicle distance, wheel torque and expected power;
the electric vehicle energy storage module is used for calculating required power and resistance under the driving and braking conditions of the electric vehicle;
the control target module ensures the safety of the vehicle by restricting the distance between the vehicles; the energy-saving driving is improved and the service life of a battery is prolonged by setting an optimization target;
the controller design module is used for determining the state variables and the specific involved contents of the cost function in the control process.
2. The electric vehicle ecological adaptive cruise control system based on reinforcement learning according to claim 1, wherein the longitudinal dynamics module comprises the following specific contents:
the controlled electric automobile and the vehicle in front of the controlled electric automobile are respectively marked as a main automobile and a front automobile; the actual vehicle distance between the main vehicle and the front vehicle is represented by L; speed of main and preceding vehicles respectivelyVhAnd VpRepresents;
a longitudinal dynamics module:
the longitudinal dynamics model of the master is represented as follows:
Figure FDA0002939020600000011
in the formula: sh(t)、vh(T) and Tω(t) position, velocity and wheel torque of the host vehicle, respectively; m, R, etatAnd delta is the main vehicle mass, the effective rolling radius of the tire, the transmission efficiency and the rotation inertia coefficient respectively; fb(t) and Fr(t) braking force and collective resistance, respectively;
lumped resistance F consisting of aerodynamic resistance, rolling resistance and gravityr(t)Fr(t) is represented as follows:
Figure FDA0002939020600000012
in the formula: phih(L(t))、Cd、μh、AvAnd theta(s)h(t)) respectively are a vehicle normalized resistance coefficient, an air resistance coefficient, a rolling resistance coefficient, a head windward area and a road surface gradient; g and rho are gravity acceleration and air density respectively; further, the distance L (t) between the host vehicle and the preceding vehicle can be expressed as
L(t)＝sp(t)-sh(t)-dcar (3)
In the formula: dcarIndicating the length of the main vehicle body, sp(t) represents a preceding vehicle position;
the torque of the wheel is output or input to the motor through the gear, and the torque T of the motormAnd a rotational speed omegamIs represented as follows:
Figure FDA0002939020600000021
in the formula: grThe fixed gear ratio of the main vehicle; wheel speed omegawThe calculation formula of (t) is as follows:
Figure FDA0002939020600000022
then, the input power of the motor inverter is given as follows:
Figure FDA0002939020600000023
in the formula: etam(t)(0＜ηm(t) < 1) represents the efficiency of the motor inverter.
3. The electric vehicle ecological adaptive cruise control system based on reinforcement learning of claim 2, characterized in that the electric vehicle energy storage system module comprises the following specific contents:
electric automobile energy storage system module:
the variable symbols are defined as follows:
·Pbat(t): the output power of the battery pack at the time t;
·Pe(t): the required power of the electric automobile at the moment t;
·Vbat(t): open circuit voltage of the battery pack at time t;
·Ibat(t): the current of the battery pack at the time t;
·SoCbat(t): the State of Charge (SOC) of the battery pack at time t;
·Rbatdisch(SoCbat(t)): the discharge resistance of the battery pack at the time t;
·Rbatch(SoCbat(t)): the charging resistance of the battery pack at the time t;
discharge resistance R of battery packbat，disch(SoCbat(t)) and a charging resistance Rbat，ch(SoCbat(t)) is represented as follows:
Figure FDA0002939020600000031
(1) a driving mode:
Figure FDA0002939020600000032
(2) regenerative braking mode:
Figure FDA0002939020600000033
the state of charge SoC of the battery is as follows:
Figure FDA0002939020600000034
4. the electric vehicle ecological adaptive cruise control system based on reinforcement learning according to claim 3, wherein the specific contents of the control target module are as follows:
a control target module:
(1) vehicle safety:
to ensure vehicle safety, constraints on the vehicle spacing are given as follows:
dmin(vh(t))≤L(t)≤dmax(vh(t)) (11)
wherein d ismin(vh(t)) and dmax(vh(t)) are respectively the minimum and maximum safe vehicle distances allowed, and their calculation formula is as follows:
Figure FDA0002939020600000035
(2) energy-saving driving:
in order to ensure the energy consumption economy of the vehicle during driving, the following optimization goals are given:
Figure FDA0002939020600000036
(3) and the service life of the battery is prolonged:
in order to reduce the battery capacity loss of the vehicle during running, the following optimization objectives are given:
Figure FDA0002939020600000037
5. the electric vehicle ecological adaptive cruise control system based on reinforcement learning according to claim 4, wherein the controller design module comprises the following specific contents:
a controller design module:
(1) bandstop function with compensation factor:
in order to obtain the error Δ d (t) between vehicles, the error of the iteration δ d (t) of the vehicle in the safety range is firstly obtained, which is specifically described as follows:
Figure FDA0002939020600000041
from the equation, Δ d (t) the inter-vehicle error can be found as:
Figure FDA0002939020600000042
wherein alpha is more than 0, beta is more than or equal to 1,
Figure FDA0002939020600000043
dmin，dmax∈R+respectively the lower and upper band stop limits, cfTo compensate forA factor;
in an optimization problem that minimizes the objective with a multi-objective cost function, the cost of the optimization is reduced when the parameters a, ss,
Figure FDA0002939020600000044
and cfAfter setting correctly, if the band stop function
Figure FDA0002939020600000045
As part of the cost function, the actual vehicle distance L (t) is limited to [ d [ [ d ]min，dmax]Within the range;
(2) demand power optimization problem based on reinforcement learning:
first, the basic variables are defined:
x (t): state variables of the electric automobile at the moment t;
·Fb(t): braking force of the electric automobile at the time t;
·ωw(t): the wheel rotating speed of the electric automobile at the time t;
·ωm(t): the motor rotating speed of the electric automobile at the moment t;
·Tm(t): the motor torque of the electric automobile at the time t;
·Tm，max(t): the maximum motor torque allowed by the electric automobile at the time t;
u (t): control input of the electric automobile at the time t;
·ηm(t): the motor efficiency of the electric automobile at the moment t;
·Pe(t): the required power of the electric automobile at the moment t;
the continuous dynamic state equation of the host at time t is as follows:
Figure FDA0002939020600000051
in the formula: x (t) ═ Δ vh(t)，Δd(t)]TA state variable representing a primary vehicle dynamics system; after defining two types of variables, optimizing the aim in the problemThe standard cost function J is as follows:
Figure FDA0002939020600000052
in the formula: u is a utility function, gamma is a discount coefficient, gamma is more than 0 and less than or equal to 1, and the function J is a cost function of the state x (t) and depends on the initial time t and the initial state x (t); the goal of reinforcement learning is to select a control sequence u (t) that minimizes the cost function defined by equation (18); in addition, the optimization goal of the objective cost function is as follows:
Figure FDA0002939020600000053
U(t)＝λiLi+λ2L2+λaLa. (20)
in the formula: considering the driving safety of the vehicle, L1With the aim of keeping the distance between the vehicles at a minimum distance dminAnd the maximum vehicle distance dmaxTo (c) to (d); in addition, the concentration of the alpha, beta,
Figure FDA0002939020600000054
and cfIs a parameter of the spacing stop band function; l is2The energy consumption economy of the vehicle during running can be improved; l is3The service life of the battery of the electric automobile can be prolonged;
assuming the expected motor torque as the control variable, the control variables optimized based on the ADHDP algorithm are given as follows
u*(·|t0)＝argminJ(x(·|t0)) (21)。
6. The electric vehicle ecological adaptive cruise control system based on reinforcement learning according to claim 5, wherein an electric vehicle ecological adaptive cruise control method based on reinforcement learning adopts an execution dependence heuristic dynamic programming framework, and comprises the following steps:
1) determining a state variable x (t) through an information acquisition module and a controller design module, determining a utility function U (t) through a control target module, and initializing relevant parameters;
2) inputting state variable x (t) into execution network acquisition control variable u (t)
3) Inputting the state variable x (t) and the control variable u (t) into an evaluation network to obtain the expected cost
Figure FDA0002939020600000055
4) Setting errors of execution network and evaluation network
5) Solving a state variable x (t +1) at the next moment through a longitudinal dynamics module and an electric vehicle energy storage module;
6) updating the weight of the execution network, inputting the state variable x (t +1) to the execution network to obtain the control variable u (t +1)
7) Updating the weight of the evaluation network, and obtaining the expected cost through the evaluation network
Figure FDA0002939020600000061
Value of
8) Judging whether the evaluation network and the execution network meet the maximum iteration times or whether the tolerance meets the self-adaptive iteration value; if the control variable u (t +1) is satisfied, the solved control variable u (t +1) is used as the optimal or suboptimal control variable, otherwise, the second step is returned.
CN202110171999.6A 2021-02-08 2021-02-08 Electric automobile ecological self-adaptation cruise control system based on reinforcement learning Active CN112896161B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110171999.6A CN112896161B (en) 2021-02-08 2021-02-08 Electric automobile ecological self-adaptation cruise control system based on reinforcement learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110171999.6A CN112896161B (en) 2021-02-08 2021-02-08 Electric automobile ecological self-adaptation cruise control system based on reinforcement learning

Publications (2)

Publication Number Publication Date
CN112896161A true CN112896161A (en) 2021-06-04
CN112896161B CN112896161B (en) 2022-06-21

Family

ID=76122736

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110171999.6A Active CN112896161B (en) 2021-02-08 2021-02-08 Electric automobile ecological self-adaptation cruise control system based on reinforcement learning

Country Status (1)

Country Link
CN (1) CN112896161B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113561976A (en) * 2021-08-19 2021-10-29 湖南大学无锡智能控制研究院 Vehicle energy-saving prediction adaptive cruise control method and device based on feedback optimization
CN113741199A (en) * 2021-09-16 2021-12-03 吉林大学 Vehicle economy speed planning method based on intelligent network connection information
CN114459498A (en) * 2022-03-14 2022-05-10 南京理工大学 New energy vehicle charging station selection and self-adaptive navigation method based on reinforcement learning
CN115424429A (en) * 2022-08-30 2022-12-02 浙江绿色慧联有限公司 ECO-CACC control method and system based on deep reinforcement learning

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020138193A1 (en) * 2001-03-22 2002-09-26 Visteon Global Technologies, Inc. Tracking of a target vehicle using adaptive cruise control
JP2007022135A (en) * 2005-07-12 2007-02-01 Toyota Motor Corp Automatic braking device of vehicle
JP2008290665A (en) * 2007-05-28 2008-12-04 Denso Corp Cruise control device, program, and setting method for target vehicle speed
US20100125399A1 (en) * 2008-11-18 2010-05-20 Bendix Commerical Vehicle Systems Llc Adaptive cruise control braking with deceleration monitoring
CN104822556A (en) * 2012-12-07 2015-08-05 大陆汽车有限公司 Method for operating recuperation brake of motor vehicle and recuperation brake
CN106740846A (en) * 2016-12-02 2017-05-31 大连理工大学 A kind of electric automobile self-adapting cruise control method of double mode switching
CN107117170A (en) * 2017-04-28 2017-09-01 吉林大学 A kind of real-time estimate cruise control system driven based on economy
CN107139923A (en) * 2017-05-11 2017-09-08 中科院微电子研究所昆山分所 A kind of ACC decision-making techniques and system
FR3060489A1 (en) * 2016-12-15 2018-06-22 Compagnie Plastic Omnium HEATING DEVICE FOR PROTECTING A MOTOR VEHICLE RADAR
CN108437991A (en) * 2018-04-11 2018-08-24 厦门大学 A kind of intelligent electric automobile adaptive cruise control system and its method
CN108528268A (en) * 2017-03-06 2018-09-14 重庆邮电大学 A kind of torque adjusting method of electric vehicle self-adaption cruise system
CN108622078A (en) * 2017-03-22 2018-10-09 丰田自动车株式会社 The control method of automobile, the control device of automobile and automobile
CN109131312A (en) * 2018-08-01 2019-01-04 厦门大学 A kind of intelligent electric automobile ACC/ESC integrated control system and its method
CN109591659A (en) * 2019-01-14 2019-04-09 吉林大学 A kind of pure electric automobile energy management control method of intelligence learning
CN109733398A (en) * 2018-12-05 2019-05-10 南京航空航天大学 Self-adaption cruise system and control method with stability active control
CN109927725A (en) * 2019-01-28 2019-06-25 吉林大学 A kind of self-adaption cruise system and implementation method with driving style learning ability
CN109969183A (en) * 2019-04-09 2019-07-05 台州学院 Bend follow the bus control method based on safely controllable domain
CN110406526A (en) * 2019-08-05 2019-11-05 合肥工业大学 Parallel hybrid electric energy management method based on adaptive Dynamic Programming
CN110871781A (en) * 2018-08-29 2020-03-10 通用汽车环球科技运作有限责任公司 Vehicle, predictive charge planning and powertrain control system and control logic

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020138193A1 (en) * 2001-03-22 2002-09-26 Visteon Global Technologies, Inc. Tracking of a target vehicle using adaptive cruise control
JP2007022135A (en) * 2005-07-12 2007-02-01 Toyota Motor Corp Automatic braking device of vehicle
JP2008290665A (en) * 2007-05-28 2008-12-04 Denso Corp Cruise control device, program, and setting method for target vehicle speed
US20100125399A1 (en) * 2008-11-18 2010-05-20 Bendix Commerical Vehicle Systems Llc Adaptive cruise control braking with deceleration monitoring
CN104822556A (en) * 2012-12-07 2015-08-05 大陆汽车有限公司 Method for operating recuperation brake of motor vehicle and recuperation brake
CN106740846A (en) * 2016-12-02 2017-05-31 大连理工大学 A kind of electric automobile self-adapting cruise control method of double mode switching
FR3060489A1 (en) * 2016-12-15 2018-06-22 Compagnie Plastic Omnium HEATING DEVICE FOR PROTECTING A MOTOR VEHICLE RADAR
CN108528268A (en) * 2017-03-06 2018-09-14 重庆邮电大学 A kind of torque adjusting method of electric vehicle self-adaption cruise system
CN108622078A (en) * 2017-03-22 2018-10-09 丰田自动车株式会社 The control method of automobile, the control device of automobile and automobile
CN107117170A (en) * 2017-04-28 2017-09-01 吉林大学 A kind of real-time estimate cruise control system driven based on economy
CN107139923A (en) * 2017-05-11 2017-09-08 中科院微电子研究所昆山分所 A kind of ACC decision-making techniques and system
CN108437991A (en) * 2018-04-11 2018-08-24 厦门大学 A kind of intelligent electric automobile adaptive cruise control system and its method
CN109131312A (en) * 2018-08-01 2019-01-04 厦门大学 A kind of intelligent electric automobile ACC/ESC integrated control system and its method
CN110871781A (en) * 2018-08-29 2020-03-10 通用汽车环球科技运作有限责任公司 Vehicle, predictive charge planning and powertrain control system and control logic
CN109733398A (en) * 2018-12-05 2019-05-10 南京航空航天大学 Self-adaption cruise system and control method with stability active control
CN109591659A (en) * 2019-01-14 2019-04-09 吉林大学 A kind of pure electric automobile energy management control method of intelligence learning
CN109927725A (en) * 2019-01-28 2019-06-25 吉林大学 A kind of self-adaption cruise system and implementation method with driving style learning ability
CN109969183A (en) * 2019-04-09 2019-07-05 台州学院 Bend follow the bus control method based on safely controllable domain
CN110406526A (en) * 2019-08-05 2019-11-05 合肥工业大学 Parallel hybrid electric energy management method based on adaptive Dynamic Programming

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113561976A (en) * 2021-08-19 2021-10-29 湖南大学无锡智能控制研究院 Vehicle energy-saving prediction adaptive cruise control method and device based on feedback optimization
CN113561976B (en) * 2021-08-19 2022-04-19 湖南大学无锡智能控制研究院 Vehicle energy-saving prediction adaptive cruise control method and device based on feedback optimization
CN113741199A (en) * 2021-09-16 2021-12-03 吉林大学 Vehicle economy speed planning method based on intelligent network connection information
CN113741199B (en) * 2021-09-16 2024-04-26 吉林大学 Whole vehicle economical speed planning method based on intelligent network connection information
CN114459498A (en) * 2022-03-14 2022-05-10 南京理工大学 New energy vehicle charging station selection and self-adaptive navigation method based on reinforcement learning
CN115424429A (en) * 2022-08-30 2022-12-02 浙江绿色慧联有限公司 ECO-CACC control method and system based on deep reinforcement learning
CN115424429B (en) * 2022-08-30 2024-06-07 浙江绿色慧联有限公司 ECO-CACC control method and system based on deep reinforcement learning

Also Published As

Publication number Publication date
CN112896161B (en) 2022-06-21

Similar Documents

Publication Publication Date Title
CN112896161B (en) Electric automobile ecological self-adaptation cruise control system based on reinforcement learning
Hu et al. Model predictive control of hybrid electric vehicles for fuel economy, emission reductions, and inter-vehicle safety in car-following scenarios
He et al. An intelligent braking system composed single-pedal and multi-objective optimization neural network braking control strategies for electric vehicle
CN105644560B (en) A kind of four-wheel wheel hub motor electric vehicle self-adapting cruise control system and method
Li et al. AMT downshifting strategy design of HEV during regenerative braking process for energy conservation
CN105857312B (en) A kind of highway heavy truck speed travels optimization method
Xiong et al. A new dual axle drive optimization control strategy for electric vehicles using vehicle-to-infrastructure communications
CN109484407A (en) A kind of adaptive follow the bus method that electric car auxiliary drives
CN106314202B (en) Sliding energy recovery method and device for pure electric vehicle and pure electric vehicle
CN113635879B (en) Vehicle braking force distribution method
CN108528451A (en) Self-adaptive cruise control method and device and electric automobile
CN110667393B (en) Recovery control method for sliding energy of pure electric vehicle
CN113459829B (en) Intelligent energy management method for double-motor electric vehicle based on road condition prediction
Wang et al. Dual-mode adaptive cruise control strategy based on model predictive control and neural network for pure electric vehicles
CN110667564B (en) Intelligent management method for autonomous queue running energy of parallel hybrid electric vehicle
Pusztai et al. Vehicle model for driving strategy optimization of energy efficient lightweight vehicle
Zhou et al. Energy optimization for intelligent hybrid electric vehicles based on hybrid system approach in a car‐following process
He The Electric Vehicle Torque Adaptive Drive Anti-Skid Control Based on Objective Optimization.
He et al. Parameter matching and simulation analysis of electromechanical coupling device for hybrid electric vehicle
CN114954455B (en) Electric vehicle following driving control method based on multi-step reinforcement learning
Qu et al. Instantaneous velocity optimization strategy of electric vehicle considering varying road slopes
Xu et al. Look-ahead Horizon based Energy Optimization for Connected Hybrid Electric Vehicles
Li Torque optimization strategy for pure electric vehicles based on drive mode recognition
Ou et al. Torque allocation strategy for two axles four wheel drive electric vehicle with improvement of economy and stability
CN116834776A (en) Vehicle multi-target cruising and energy management control method, system and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant