CN111738611B - Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm - Google Patents
Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm Download PDFInfo
- Publication number
- CN111738611B CN111738611B CN202010607160.8A CN202010607160A CN111738611B CN 111738611 B CN111738611 B CN 111738611B CN 202010607160 A CN202010607160 A CN 202010607160A CN 111738611 B CN111738611 B CN 111738611B
- Authority
- CN
- China
- Prior art keywords
- charging pile
- mobile charging
- battery
- bat
- mobile
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000002787 reinforcement Effects 0.000 claims abstract description 19
- 230000032683 aging Effects 0.000 claims abstract description 17
- 230000006870 function Effects 0.000 claims abstract description 16
- 230000005611 electricity Effects 0.000 claims abstract description 10
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 claims description 28
- 229910052744 lithium Inorganic materials 0.000 claims description 28
- 230000009471 action Effects 0.000 claims description 11
- 238000007599 discharging Methods 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 8
- 230000008901 benefit Effects 0.000 claims description 6
- 238000004146 energy storage Methods 0.000 claims description 4
- 230000000875 corresponding effect Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013101 initial test Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06315—Needs-based resource requirements planning or analysis
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
- B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
- B60L53/30—Constructional details of charging stations
- B60L53/31—Charging columns specially adapted for electric vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
- B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
- B60L53/60—Monitoring or controlling charging stations
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
- B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
- B60L53/60—Monitoring or controlling charging stations
- B60L53/64—Optimising energy costs, e.g. responding to electricity rates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/60—Other road transportation technologies with climate change mitigation effect
- Y02T10/70—Energy storage systems for electromobility, e.g. batteries
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/60—Other road transportation technologies with climate change mitigation effect
- Y02T10/7072—Electromobility specific charging systems or methods for batteries, ultracapacitors, supercapacitors or double-layer capacitors
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
- Y02T90/10—Technologies relating to charging of electric vehicles
- Y02T90/12—Electric charging stations
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Power Engineering (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Marketing (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Charge And Discharge Circuits For Batteries Or The Like (AREA)
- Electric Propulsion And Braking For Vehicles (AREA)
Abstract
The invention discloses a mobile charging pile group intelligent scheduling method based on a Sarsa algorithm, which comprises the following steps of: the aggregator synthesizes the current state and the real-time electricity price of each mobile charging pile and puts forward the power requirement of the mobile charging pile group for participating in auxiliary service; establishing an aging model of a battery used for the mobile charging pile; designing a loss cost objective function and constraint conditions of the mobile charging pile battery; acquiring a day-ahead plan of the mobile charging pile through an aggregator; and the charging pile automatically moves to the electric vehicle charging area and the electric power market auxiliary service area through a dispatching instruction of the Sarsa reinforcement learning algorithm. The invention designs the model of the whole dispatching system and corresponding constraint conditions, and the whole dispatching system has learning capacity through the use of reinforcement learning, so that the mobile charging pile actively participates in the auxiliary service of the electric power market on the premise of meeting the daily charging service of the mobile charging pile.
Description
Technical Field
The invention belongs to the field of application of charging devices, and relates to an intelligent dispatching method for a mobile charging pile group based on a Sarsa algorithm.
Background
The flexibility of filling the electric pile can be effectively improved by moving the electric pile, the coupling between the parking space and the electric pile is avoided, when the electric quantity of a new energy automobile is full, the next task is started when the electric pile is idle, and the electric pile has higher use efficiency. The mobile charging pile still has the capacity of participating in auxiliary service of the electric power market in idle time of vehicle charging, and the novel application value is brought to the mobile charging pile. The unified scheduling of the idle mobile charging piles in a certain area is an important means for realizing and exerting all commercial values of the mobile charging piles. How to meet the power requirement of the electric auxiliary service by designing an optimized scheduling algorithm and simultaneously minimize the operation cost of the charging pile group has important significance for the system operation in the mode.
The mobile charging pile consists of an energy storage device and a matched mobile device, and at present, the research on the intelligent scheduling method of the mobile charging pile cluster is still relatively few. To implement intelligent scheduling of mobile charging pile clusters, a system model of the operation mode must be first established; on the basis, a corresponding objective function is established, and constraint conditions are set; and adopting a corresponding optimization algorithm, and solving to realize the optimal scheduling of the cluster device. However, the actual application of the current cluster scheduling algorithm depends on an accurate physical model, and the pre-established physical model presents larger uncertainty along with the change of the real-time environment.
Disclosure of Invention
Aiming at the defects in the prior art, the invention stands on the standpoint of a mobile charging pile group, takes the power requirement meeting the electric auxiliary service as a constraint condition, and takes the minimum scheduling loss cost of the mobile charging pile as an objective function, and provides an intelligent scheduling method for the mobile charging pile group based on a Sarsa algorithm. Firstly, an operation model of a mobile charging pile system is established, and corresponding loss cost objective functions and constraint conditions are designed; and acquiring daily scheduling of the mobile charging piles by an aggregator, and further optimizing scheduling strategies in a short time scale in a reinforcement learning manner to realize intelligent scheduling of the mobile charging pile clusters.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
A mobile charging pile group intelligent scheduling method based on a Sarsa algorithm comprises the following steps:
the aggregator synthesizes the current state and the real-time electricity price of each mobile charging pile and puts forward the power requirement of the mobile charging pile group for participating in auxiliary service;
establishing an aging model of a battery used for the mobile charging pile;
designing a loss cost objective function and constraint conditions of the mobile charging pile battery;
acquiring a day-ahead plan of the mobile charging pile through an aggregator;
and the charging pile automatically moves to the electric vehicle charging area and the electric power market auxiliary service area through a dispatching instruction of the Sarsa reinforcement learning algorithm.
In order to optimize the technical scheme, the specific measures adopted further comprise:
Further, an accelerated aging test is adopted for the lithium battery of the mobile charging pile, the relation between the capacity attenuation and the cycle number of the power lithium battery is obtained, and the specific result is fitted by adopting the following formula:
Wherein C Bat is the capacity of the battery, n c is the number of cycles, and K a is the coefficient to be fitted;
The percentage of lithium battery decay was calculated according to the following formula:
Wherein, C Eol is the corresponding capacity when the service life of the battery is cut off, and C init is the initial capacity of the battery;
According to the initial cost of the battery being spread to all working conditions, the method is as follows:
Wherein E Bat is the use cost of the lithium battery in a specific working condition, N Bat is the total number of the lithium batteries involved, and E Bat_ini is the initial investment cost of the lithium battery.
Further, the cycle times of the lithium battery are calculated by a rain flow counting method according to the current actual operation conditions.
Further, the same lithium battery is used for the mobile charging pile to carry out charging and discharging tests to obtain loss cost:
At least 5 different multiplying powers are selected for constant current charge and discharge, and the relationship between the charge and discharge efficiency and the power of the battery is obtained through fitting respectively, as shown in the following formula,
ηch＝a·Ich+b
ηdis＝c·Idis+d
Wherein η ch is charging efficiency, η dis is discharging efficiency, I ch is charging current, I dis is discharging current, and a, b, c, d is fitting parameter;
the charging and discharging loss of the obtained movable charging pile is as follows:
Lch＝r·Ich·Ubat·(1-ηch)·t
Where U bat is the terminal voltage of the battery, L ch is the charge loss cost, L dis is the discharge loss cost, r is the electricity price, and t is the duration.
Further, an objective function of minimizing loss cost for the mobile charging pile group is set as:
min J＝EBat+LBat
LBat＝Lch+Ldis
Wherein E Bat is the ageing cost, L Bat is the charge-discharge loss cost, min is the minimum value, and J is the objective function.
Further, the power of the mobile charging pile group satisfies the following constraint:
wherein M is the number of mobile charging piles, P command provides a power reference value of auxiliary service for the power of the ith charging pile,/>, andThe maximum value of the power that can be provided for the ith charging peg.
Further, the employed Sarsa algorithm structure includes: intelligent scheduling decision, environment, action, observation value and rewarding five elements; the environment is the dispatching of the movable charging pile group by the aggregator, the Sarsa algorithm observes the corresponding result from the environment by dispatching the specific actions of the movable charging pile, calculates rewards with profit as the target, and the optimal dispatching method is the maximum profit obtained by the intelligent dispatching method.
Further, the unified scheduling step of the reinforcement learning mobile charging pile using the Sarsa algorithm is as follows:
Step1, randomly initializing a Q table Q (s t,at), wherein the Q table is a table for selecting rewards obtained during different actions, and simultaneously initializing a learning rate alpha and a discount factor gamma, wherein the alpha is a number smaller than 1, and the gamma is between 0 and 1; taking a Q table containing two actions, a 1 and a 2, s1, s2 as an example, the Q table may represent rewards recorded by each row of the table for executing the corresponding action in the current state as follows.
Step2, randomly selecting a dispatching method a t from a Q table by adopting an epsilon-greedy mode, and observing system benefits R t and a next state S i,t;
step3, updating the Q table in the following way:
Q(st,at)＝Q(st,at)+α·[Rt+γ·Q(s′t,a′t)-Q(st,at)]
Step4. Repeating steps 2 to Step3 until the state S i,t reaches the target expected value S t, and ending the algorithm.
Further, when the mobile charging pile charges the electric automobile or is in a standby state, the mobile charging pile is in an operation mode; outside the operating mode, the mobile charging stake is in idle mode, can be used to participate in electric market assistance services.
Further, the mobile charging pile is a charging device capable of moving autonomously and comprises an energy storage unit, a moving device and a power electronic device.
The beneficial effects of the invention are as follows: the invention provides a method for obtaining the minimum loss cost of the movable charging pile group through a Sarsa reinforcement learning algorithm. The model of the whole dispatching system and corresponding constraint conditions are designed, the whole dispatching system has learning capacity through the use of reinforcement learning, the actual application is driven continuously through data to gradually tend to be optimized, and the excessive dependence on an accurate physical model is avoided. According to the invention, the mobile charging pile actively participates in the auxiliary service of the electric power market on the premise of meeting the daily charging service of the mobile charging pile, and benefits are obtained from the auxiliary service, so that the win-win mode of the regional power grid and the mobile charging pile operator can be realized.
Drawings
FIG. 1 is a schematic diagram of steps of an intelligent scheduling method of the present invention.
Fig. 2 is a schematic diagram of the main structure of a mobile charging pile according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a system architecture of a mobile charging pile participating in an auxiliary service of an electric power market according to an embodiment of the present invention.
FIG. 4 is a schematic diagram of the main structure of reinforcement learning in one embodiment of the present invention.
Detailed Description
The invention will now be described in further detail with reference to the accompanying drawings.
It should be noted that the terms like "upper", "lower", "left", "right", "front", "rear", and the like are also used for descriptive purposes only and are not intended to limit the scope of the invention in which the invention may be practiced, but rather the relative relationship of the terms may be altered or modified without materially altering the teachings of the invention.
According to the daily scheduling planning of the aggregator platform, on the basis, in view of the fact that the daily situation is complex, the real-time physical model is difficult to accurately establish, the problem of optimal scheduling of the mobile charging pile clusters in a short time scale is solved by the reinforced learning algorithm Sarsa in the artificial intelligence field in a data driving mode, and therefore intelligent scheduling of the mobile charging pile participating in auxiliary service of the electric power market is achieved.
The mobile charging pile is an autonomously movable charging device which is composed of an energy storage unit, a mobile device, a power electronic device and the like. The mobile charging pile is used for responding to the requirement, can autonomously move to a new energy vehicle to complete charging, and also can autonomously move to an auxiliary service area to participate in auxiliary service of an electric power market.
The intelligent scheduling method and the system realize intelligent scheduling of the mobile charging pile group in a mode of an aggregator (a cloud platform responsible for operation of the mobile charging pile). And the electricity selling company clears and distributes dynamic electricity prices according to the running condition of the regional power grid, and the cloud end aggregate synthesizes the current state of each mobile charging pile and the real-time electricity prices and proposes the power requirements of the mobile charging pile group for participating in auxiliary services. On the basis, the mobile charging pile group autonomously moves to the electric vehicle charging area and the electric power market auxiliary service area through a scheduling instruction based on a Sarsa reinforcement learning algorithm. The main objective of the invention is to minimize the loss cost of a mobile charging pile group.
As shown in fig. 1-4, the present invention includes the steps of: the aggregator synthesizes the current state and the real-time electricity price of each mobile charging pile and puts forward the power requirement of the mobile charging pile group for participating in auxiliary service;
establishing an aging model of a battery used for the mobile charging pile;
1) And (5) ageing cost of the mobile charging pile battery. In certain mobile charging pile application products, the lithium battery used has been determined. In order to obtain an aging model which is closer to the actual attenuation characteristic of the power lithium battery, the embodiment of the invention directly adopts an accelerated aging test on the same type of lithium battery of the mobile charging pile, and the used working condition is a circulation working condition for providing auxiliary service for the mobile charging pile. According to the long-period aging test result, the relation between the capacity attenuation and the cycle number of the power lithium battery can be obtained, and the specific result can be fitted by adopting the following formula:
Wherein, C Bat is the capacity of the battery, n c is the cycle number, and K a is the coefficient to be fitted.
The cycle times of the lithium battery can be calculated and obtained by a rain flow counting method according to the current actual operation conditions. The use of rain flow counting to count DoD (Depth of Discharge ) specifically includes: (1) setting an initial value SOC value to be a maximum value; (2) The rain flow flows downwards from the inner side of each SOC peak value in sequence, and falls down at the next SOC peak value until a SOC peak value larger than the starting point of the next SOC peak value is opposite; (3) Stopping immediately when the rain flow meets the rain flow under the upper layer flow; the difference between the start and end points of each stream obtained is defined as the range of variation of DOD.
On this basis, the cycle number n c is obtained by using the relation n c =f (DoD) between DoD and the maximum charge-discharge cycle number in the battery manual. f (DOD) can be obtained by offline fitting after actual aging test of the lithium battery.
From the calculated current battery capacity C Bat, the percentage of lithium battery decay can be calculated according to the following formula:
Where C Bat is the current battery capacity, C Eol is the corresponding capacity at the end of battery life, and C init is the initial battery capacity.
On the basis of obtaining the attenuation percentage of the lithium battery along with working conditions, initial test cost of the battery can be evenly spread to all working conditions according to the following formula:
Wherein E Bat is the use cost of the lithium battery in a specific working condition, N Bat is the total number of the lithium batteries involved, and E Bat_ini is the initial investment cost of the lithium battery.
2) And the charge and discharge loss cost of the mobile charge pile battery. In order to obtain the charge and discharge efficiency of the power lithium battery which is closer to the actual use of the mobile charge pile, the same type of lithium battery for the mobile charge pile is subjected to charge and discharge test. At least 5 different multiplying powers are selected for constant current charge and discharge, and the relationship between the charge and discharge efficiency and the power of the battery is obtained through fitting respectively, as shown in the following formula,
ηch＝a·Ich+b (4)
ηdis＝c·Idis+d (5)
Where η ch is the charging efficiency, η dis is the discharging efficiency, I ch is the charging current, I dis is the discharging current, and a, b, c, d is the fitting parameter.
Therefore, the charging and discharging loss of the movable charging pile is as follows:
Lch＝r·Ich·Ubat·(1-ηch)·t (6)
Where U bat is the terminal voltage of the battery, L ch is the charge loss cost, L dis is the discharge loss cost, r is the electricity price, and t is the duration.
3) And a unified scheduling model of the mobile charging pile cluster. The objective function of minimizing the loss cost of the mobile charging pile group is set as follows:
min J＝EBat+LBat (8)
Wherein E Bat is the aging cost, L Bat is the charge-discharge loss cost, which can be obtained by accumulation of charge-discharge loss, i.e. L Bat＝Lch+Ldis, min is the minimum value, and J is the objective function.
In order to meet the need to participate in the auxiliary services of the electric power market, the power of the mobile charging pile group should satisfy the following constraints:
wherein M is the number of mobile charging piles, P command provides a power reference value of auxiliary service for the power of the ith charging pile,/>, andThe maximum value of the power that can be provided for the ith charging peg.
4) And intelligent dispatching of the mobile charging pile clusters. The invention completes daily real-time scheduling of the mobile charging pile based on reinforcement learning Sarsa. Compared with the traditional optimal scheduling algorithm based on the accurate physical model, the method has the advantages that reinforcement learning can be directly driven by data, and the optimal scheduling result is obtained through multiple iterations. Because various uncertainty factors exist in the scheduling process of the short time scale, the optimal scheduling strategy is directly obtained by data by using reinforcement learning, and the deviation caused by uncertain models in the traditional method can be avoided. The reinforcement learning main structure used is shown in fig. 4, and includes: intelligent scheduling decision, environment, action, observation value, rewarding five elements. The environment related by the invention is the dispatching of the movable charging pile group by the aggregator, the Sarsa algorithm observes the corresponding result Q t from the environment by dispatching the specific action a t of the movable charging pile, the reward R t is calculated with the profit as the target, and the optimal dispatching strategy is the maximum benefit obtained by the intelligent dispatching method.
Defining reinforcement learning reward R t as an objective function (8) already defined, and state S t as charge and discharge power required for satisfying electric power market auxiliary service in daily planning, the unified scheduling step of reinforcement learning using Sarsa algorithm for short time scale mobile charging pile is as follows:
Step1, randomly initializing a Q table Q (s t,at), wherein the Q table is a table for selecting rewards obtained during different actions, and initializing a learning rate alpha and a discount factor gamma, wherein the value of alpha is usually less than 1, and the value of gamma is between 0 and 1. Taking a Q table containing two actions, a 1 and a 2, s1, s2 as an example, the Q table may represent rewards recorded by each row of the table for executing the corresponding action in the current state as follows.
Step2, randomly selecting a scheduling strategy a t from the Q table by adopting an epsilon-greedy mode, and observing system benefits R t and a next state S i,t;
step3, updating the Q table in the following way:
Q(st,at)＝Q(st,at)+α·[Rt+γ·Q(s′t,a′t)-Q(st,at)] (11)
Step4. Repeating steps 2 to Step3 until the state S i,t reaches the target expected value S t, and ending the algorithm.
The intelligent scheduling optimization result of the mobile charging pile group within 30 minutes can be obtained through the reinforcement learning algorithm based on the Sarsa, and the real-time scheduling result in practical application is based on the result of the Sarsa algorithm.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the invention without departing from the principles thereof are intended to be within the scope of the invention as set forth in the following claims.
Claims (5)
1. The intelligent scheduling method for the mobile charging pile group based on the Sarsa algorithm is characterized by comprising the following steps of:
the aggregator synthesizes the current state and the real-time electricity price of each mobile charging pile and puts forward the power requirement of the mobile charging pile group for participating in auxiliary service;
Establishing an aging model of a battery used for the mobile charging pile, and obtaining the aging cost and the charge-discharge loss cost of the battery of the mobile charging pile;
According to the aging cost and the charge-discharge loss cost of the mobile charging pile battery, designing a loss cost objective function and constraint conditions of mobile charging pile battery scheduling;
acquiring a mobile charging pile meeting the objective function and a corresponding power reference value for providing auxiliary service according to the scheduled loss cost objective function and the constraint condition; acquiring a day-ahead schedule of the mobile charging pile by an aggregator based on the mobile charging pile meeting the objective function and a corresponding power reference value for providing auxiliary service;
The scheduling strategy is further optimized in a short time scale in a reinforcement learning mode of a Sarsa algorithm; the Sarsa algorithm structure comprises: intelligent scheduling decision, environment, action, observation value and rewarding five elements; the environment is the dispatching of the movable charging pile group by the aggregator, the Sarsa algorithm observes the corresponding result from the environment by dispatching the specific actions of the movable charging pile, calculates rewards with profit as a target, and the optimal dispatching method is the maximum profit obtained by the intelligent dispatching method;
the charging pile moves to the electric automobile charging area and the electric power market auxiliary service area independently through a dispatching instruction of a Sarsa reinforcement learning algorithm;
And (3) adopting an accelerated aging test on the lithium battery of the mobile charging pile to obtain the relation between the capacity attenuation and the cycle number of the power lithium battery, and fitting the specific result by adopting the following formula:
Wherein C Bat is the capacity of the battery, n c is the number of cycles, and a is the coefficient to be fitted;
The percentage of lithium battery decay was calculated according to the following formula:
Wherein, C Eol is the corresponding capacity when the service life of the battery is cut off, and C init is the initial capacity of the battery;
According to the initial cost of the battery being spread to all working conditions, the method is as follows:
wherein E Bat is the aging cost, N Bat is the total number of lithium batteries involved, and E Bat_ini is the initial investment cost of the lithium batteries;
The same type lithium battery is used for charging and discharging test on the mobile charging pile to obtain loss cost:
At least 5 different multiplying powers are selected for constant current charge and discharge, and the relationship between the charge and discharge efficiency and the power of the battery is obtained through fitting respectively, as shown in the following formula,
ηch＝a·Ich+b
ηdis＝c·Idis+d
Wherein η ch is charging efficiency, η dis is discharging efficiency, I ch is charging current, I dis is discharging current, and a, b, c, d is fitting parameter;
the charging and discharging loss of the obtained movable charging pile is as follows:
Lch＝r·Ich·Ubat·(1-ηch)·t
Wherein U bat is the terminal voltage of the battery, L ch is the charge loss cost, L dis is the discharge loss cost, r is the electricity price, and t is the duration;
the objective function of minimizing the loss cost of the mobile charging pile group is set as follows:
min J＝EBat+LBat
LBat＝Lch+Ldis
Wherein E Bat is the ageing cost, L Bat is the charge-discharge loss cost, min is the minimum value, and J is the objective function;
the power of the mobile charging pile group satisfies the following constraint:
wherein M is the number of mobile charging piles, For the power of the ith charging pile, P command provides a power reference value for auxiliary services,/>, forThe maximum value of the power that can be provided for the ith charging peg.
2. The intelligent scheduling method for the mobile charging pile group according to claim 1, wherein the cycle number of the lithium battery is calculated by a rain flow counting method according to the current actual operation condition.
3. The intelligent scheduling method for mobile charging pile groups according to claim 1, wherein the unified scheduling step of the reinforcement learning mobile charging pile using the Sarsa algorithm is as follows:
Step1, randomly initializing a Q table Q (s t,at), wherein the Q table is a table for selecting rewards obtained during different actions, and simultaneously initializing a learning rate alpha and a discount factor gamma, wherein the alpha is a number smaller than 1, and the gamma is between 0 and 1; step2, randomly selecting a dispatching method a t from a Q table by adopting an epsilon-greedy mode, and observing system benefits R t and a next state S i,t;
step3, updating the Q table in the following way:
Q(st,at)＝Q(st,at)+α·[Rt+γ·Q(s't,a't)-Q(st,at)]
Step4. Repeating steps 2 to Step3 until the state S i,t reaches the target expected value S t, and ending the algorithm.
4. The intelligent dispatching method of mobile charging pile groups according to claim 1, wherein when the mobile charging pile charges an electric vehicle or is in a standby state, the mobile charging pile is in an operation mode; outside the operating mode, the mobile charging stake is in idle mode, can be used to participate in electric market assistance services.
5. The intelligent dispatching method for mobile charging piles according to claim 1, wherein the mobile charging piles are charging devices capable of moving autonomously, and the intelligent dispatching method comprises an energy storage unit, a mobile device and a power electronic device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010607160.8A CN111738611B (en) | 2020-06-29 | 2020-06-29 | Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010607160.8A CN111738611B (en) | 2020-06-29 | 2020-06-29 | Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111738611A CN111738611A (en) | 2020-10-02 |
| CN111738611B true CN111738611B (en) | 2024-04-26 |
Family
ID=72652775
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010607160.8A Active CN111738611B (en) | 2020-06-29 | 2020-06-29 | Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111738611B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112590598B (en) * | 2020-12-11 | 2023-11-07 | 合肥工业大学 | Optimal configuration method and system for mobile charging vehicle |
| CN116629413B (en) * | 2023-05-10 | 2024-06-21 | 金陵科技学院 | Global optimal automatic deployment method for mobile charging device based on random forest and Dijkstra |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160030643A (en) * | 2014-09-11 | 2016-03-21 | 한밭대학교 산학협력단 | Apparatus and method for real-time estimation of battery state-of-health |
| CN106469354A (en) * | 2016-09-09 | 2017-03-01 | 中国电力科学研究院 | A kind of user's request response participatory approaches under Load aggregation quotient module formula |
| CN207560060U (en) * | 2017-12-26 | 2018-06-29 | 数源科技股份有限公司 | Intelligent mobile charging pile group system |
| EP3386059A1 (en) * | 2017-04-06 | 2018-10-10 | ABB Schweiz AG | Method and system for providing ancillary grid services |
| CN109510250A (en) * | 2018-12-19 | 2019-03-22 | 洁电（北京）储能科技有限公司 | Charging station, charging pile system and control method with power grid ancillary service function |
| CN109816171A (en) * | 2019-01-28 | 2019-05-28 | 三峡大学 | A kind of electric car region microgrid group's bilayer distributed optimization dispatching method based on fuzzy Spot Price |
| CN109910671A (en) * | 2019-03-11 | 2019-06-21 | 三峡大学 | Electric car V2G control method based on intelligent charging spot |
| CN110281809A (en) * | 2019-05-28 | 2019-09-27 | 北京航盛新能科技有限公司 | A kind of electric car V2G control system based on particle swarm optimization algorithm |
| CN111016725A (en) * | 2019-12-31 | 2020-04-17 | 西安交通大学 | Mode selection control method of energy storage type charging pile |
| CN111106612A (en) * | 2018-10-25 | 2020-05-05 | 贵州电网有限责任公司电力科学研究院 | Energy storage type charging pile participating power grid demand side response combined operation optimization model and solving algorithm |
| CN210502331U (en) * | 2019-07-17 | 2020-05-12 | 深圳市瑞能实业股份有限公司 | Energy storage charging pile system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105046371A (en) * | 2015-08-19 | 2015-11-11 | 东南大学 | Electric vehicle charge-discharge scheduling method based on demand side bidding |
| CN110210709B (en) * | 2019-04-30 | 2023-04-18 | 太原理工大学 | Frequency modulation instruction scheduling method considering life attenuation and capacity balance bidding of energy storage power station |
-
2020
- 2020-06-29 CN CN202010607160.8A patent/CN111738611B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160030643A (en) * | 2014-09-11 | 2016-03-21 | 한밭대학교 산학협력단 | Apparatus and method for real-time estimation of battery state-of-health |
| CN106469354A (en) * | 2016-09-09 | 2017-03-01 | 中国电力科学研究院 | A kind of user's request response participatory approaches under Load aggregation quotient module formula |
| EP3386059A1 (en) * | 2017-04-06 | 2018-10-10 | ABB Schweiz AG | Method and system for providing ancillary grid services |
| CN207560060U (en) * | 2017-12-26 | 2018-06-29 | 数源科技股份有限公司 | Intelligent mobile charging pile group system |
| CN111106612A (en) * | 2018-10-25 | 2020-05-05 | 贵州电网有限责任公司电力科学研究院 | Energy storage type charging pile participating power grid demand side response combined operation optimization model and solving algorithm |
| CN109510250A (en) * | 2018-12-19 | 2019-03-22 | 洁电（北京）储能科技有限公司 | Charging station, charging pile system and control method with power grid ancillary service function |
| CN109816171A (en) * | 2019-01-28 | 2019-05-28 | 三峡大学 | A kind of electric car region microgrid group's bilayer distributed optimization dispatching method based on fuzzy Spot Price |
| CN109910671A (en) * | 2019-03-11 | 2019-06-21 | 三峡大学 | Electric car V2G control method based on intelligent charging spot |
| CN110281809A (en) * | 2019-05-28 | 2019-09-27 | 北京航盛新能科技有限公司 | A kind of electric car V2G control system based on particle swarm optimization algorithm |
| CN210502331U (en) * | 2019-07-17 | 2020-05-12 | 深圳市瑞能实业股份有限公司 | Energy storage charging pile system |
| CN111016725A (en) * | 2019-12-31 | 2020-04-17 | 西安交通大学 | Mode selection control method of energy storage type charging pile |
Non-Patent Citations (10)
| Title |
|---|
| A load Scheduling Strategy for Electric Vehicles Charging System;Zheng Wang 等;《2014 IEEEEIGHTH INTERNATIONAL CONFERENCE ON SOFTWARE SECURITY AND RELIABILITY-COMPANION(SERE-C 2014)》;218-222 * |
| Joint Optimization of Delay-Tolerant Autonomous Electric Vehicles Charge Scheduling and Station Bateery Degradation;Yongsheng Cao 等;《IEEE Internet of Things Joural》;第7卷(第9期);8590-859 * |
| V2G充电桩与智能微电网的并网技术研究;曹括;中国优秀硕士学位论文全文数据库;工程科技Ⅱ辑(第2019年09期);C034-247 * |
| 基于V2G技术的电动汽车充电桩的充放电控制策略研究;裴荣娜;中国优秀硕士学位论文全文数据库;工程科技Ⅱ辑(第2017年02期);C034-1159 * |
| 智能电网中多储能装置的学习优化控制;吕凯;中国优秀硕士学位论文全文数据库;工程科技Ⅱ辑(第2016年第06期);C042-591 * |
| 电动汽车聚合服务商参与电网辅助服务的调控策略研究;赵亚杰;中国优秀硕士学位论文全文数据库;工程科技Ⅱ辑(第2020年01期);C042-86 * |
| 考虑寿命的电动汽车动力电池参与电网调频策略研究;姜伊;中国优秀硕士学位论文全文数据库;工程科技Ⅱ辑(第2017年02期);C035-786 * |
| 考虑电动汽车的并网型微电网优化协调调度;陈浩宇;中国优秀硕士学位论文全文数据库;工程科技Ⅱ辑(第2019年第04期);C035-360 * |
| 锂离子电池寿命预测研究;王宁;中国优秀硕士学位论文全文数据库;工程科技Ⅱ辑(第2019年08期);C042-1189 * |
| 面向电力市场的分布式储能聚合参与电网调峰;林立乾 等;储能科学与技术(第201902期);276-283 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111738611A (en) | 2020-10-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738611B (en) | Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm | |
| CN111137168B (en) | Battery charging method and system for battery replacement station | |
| CN109217290B (en) | Microgrid energy optimization management method considering electric vehicle charging and discharging | |
| CN109193721B (en) | Electric vehicle charging and discharging strategy optimization method based on reinforcement learning | |
| CN113176511B (en) | Energy storage charging and discharging optimization method and system considering health state | |
| CN108596667B (en) | Electric automobile real-time charging electricity price calculation method based on Internet of vehicles | |
| CN110011342A (en) | A kind of cluster electric car charge-discharge electric power optimum management method | |
| CN109193718A (en) | A kind of selection electric car networking regulation method being adapted to V2G | |
| CN111833205B (en) | Intelligent scheduling method for mobile charging pile group under big data scene | |
| CN107240729B (en) | Method for charging batteries | |
| CN111244991A (en) | Energy storage power station operation control method and system considering battery life | |
| CN104156789A (en) | Isolated micro-grid optimum economic operation method taking energy storage life loss into consideration | |
| CN108964101A (en) | The construction method and device of application scenarios model coexists in a kind of V2B and V2G | |
| CN113659623B (en) | Optimization method and system of wind-storage combined system based on Boolean line theory | |
| CN109523079B (en) | Optimization method for combined auxiliary peak shaving by utilizing electric automobile and heat accumulating electric boiler | |
| CN111724080A (en) | Mobile charging pile group scheduling method considering battery pack health state balance | |
| CN111680829A (en) | Distributed energy storage cluster scheduling method considering charge and discharge cost | |
| CN113495212A (en) | Method for estimating maintenance regeneration times and cycle life of maintainable regenerative battery | |
| CN110466388A (en) | A kind of composite energy storage electric automobile energy management method | |
| Ye et al. | Optimal control strategy for plug-in electric vehicles based on reinforcement learning in distribution networks | |
| CN115001053A (en) | V2G optimal frequency modulation method for actively inhibiting battery aging | |
| CN110341537A (en) | A kind of vehicle-mounted bidirectional charger charge control strategy based on Model Predictive Control | |
| CN106936145B (en) | Life optimization control method for energy storage power station | |
| CN111740417B (en) | Combined micro-grid operation method considering electric automobile charging failure | |
| Su et al. | Cost minimization control for smart electric vehicle car parks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |