CN111738611A - Mobile charging pile group intelligent scheduling method based on Sarsa algorithm - Google Patents
Mobile charging pile group intelligent scheduling method based on Sarsa algorithm Download PDFInfo
- Publication number
- CN111738611A CN111738611A CN202010607160.8A CN202010607160A CN111738611A CN 111738611 A CN111738611 A CN 111738611A CN 202010607160 A CN202010607160 A CN 202010607160A CN 111738611 A CN111738611 A CN 111738611A
- Authority
- CN
- China
- Prior art keywords
- charging pile
- mobile charging
- bat
- battery
- mobile
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007600 charging Methods 0.000 title claims abstract description 139
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000002787 reinforcement Effects 0.000 claims abstract description 19
- 230000032683 aging Effects 0.000 claims abstract description 14
- 230000006870 function Effects 0.000 claims abstract description 13
- 230000005611 electricity Effects 0.000 claims abstract description 9
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 claims description 29
- 229910052744 lithium Inorganic materials 0.000 claims description 29
- 230000009471 action Effects 0.000 claims description 11
- 238000007599 discharging Methods 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 8
- 238000004146 energy storage Methods 0.000 claims description 4
- 238000010277 constant-current charging Methods 0.000 claims description 3
- 230000000875 corresponding effect Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013101 initial test Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06315—Needs-based resource requirements planning or analysis
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
- B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
- B60L53/30—Constructional details of charging stations
- B60L53/31—Charging columns specially adapted for electric vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
- B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
- B60L53/60—Monitoring or controlling charging stations
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
- B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
- B60L53/60—Monitoring or controlling charging stations
- B60L53/64—Optimising energy costs, e.g. responding to electricity rates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/60—Other road transportation technologies with climate change mitigation effect
- Y02T10/70—Energy storage systems for electromobility, e.g. batteries
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/60—Other road transportation technologies with climate change mitigation effect
- Y02T10/7072—Electromobility specific charging systems or methods for batteries, ultracapacitors, supercapacitors or double-layer capacitors
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
- Y02T90/10—Technologies relating to charging of electric vehicles
- Y02T90/12—Electric charging stations
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Mechanical Engineering (AREA)
- Transportation (AREA)
- Power Engineering (AREA)
- Tourism & Hospitality (AREA)
- Health & Medical Sciences (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Operations Research (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Charge And Discharge Circuits For Batteries Or The Like (AREA)
- Electric Propulsion And Braking For Vehicles (AREA)
Abstract
The invention discloses a mobile charging pile group intelligent scheduling method based on a Sarsa algorithm, which comprises the following steps of: the aggregator integrates the current state and the real-time electricity price of each mobile charging pile and puts forward the power demand of the mobile charging pile group for participating in auxiliary services; establishing an aging model for a battery used by the mobile charging pile; designing a loss cost objective function and constraint conditions of the mobile charging pile battery; obtaining a day-ahead plan of the mobile charging pile through an aggregator; and the charging pile automatically moves to an electric automobile charging area and an electric power market auxiliary service area through a scheduling instruction of a Sarsa reinforcement learning algorithm. The invention designs a model of the whole dispatching system and corresponding constraint conditions, and enables the whole dispatching system to have learning capacity through the use of reinforcement learning, so that the mobile charging pile actively participates in the electric power market auxiliary service on the premise of meeting the daily charging service of the mobile charging pile.
Description
Technical Field
The invention belongs to the field of charging device application, and relates to a mobile charging pile group intelligent scheduling method based on a Sarsa algorithm.
Background
The flexibility that electric pile can effectively be filled in the removal, has avoided the parking stall and has filled the coupling between the electric pile, can fill up when new energy automobile electric quantity is full of, when filling electric pile and idle, opens next task, has higher availability factor. The mobile charging pile still has the capacity of participating in auxiliary service of the electric power market in the idle time of vehicle charging, and therefore a brand-new application value is brought to the mobile charging pile. Unified scheduling is carried out to idle mobile charging stake in a certain area, is the important means of realizing and exerting all commercial values of mobile charging stake. How to meet the power requirement of the power auxiliary service by designing an optimized scheduling algorithm and simultaneously minimizing the operation cost of a charging pile group has important significance for system operation in the mode.
The mobile charging pile is composed of an energy storage device and a matched mobile device, and at present, researches on the mobile charging pile cluster intelligent scheduling method are still relatively few. To realize the intelligent scheduling of the mobile charging pile cluster, a system model of the operation mode must be established firstly; on the basis, establishing a corresponding objective function and setting constraint conditions; and adopting a corresponding optimization algorithm to realize the optimized scheduling of the cluster device through solving. However, in practical application of the current cluster scheduling algorithm, the current cluster scheduling algorithm mostly depends on an accurate physical model, and a pre-established physical model presents a large uncertainty along with the change of a real-time environment.
Disclosure of Invention
Aiming at the defects in the prior art, the method stands on the standpoint of a mobile charging pile group, takes the power requirement meeting the power auxiliary service as a constraint condition and takes the minimum scheduling loss cost of the mobile charging pile as an objective function, and provides an intelligent scheduling method for the mobile charging pile group based on the Sarsa algorithm. Firstly, establishing an operation model of a mobile charging pile system, and designing a corresponding loss cost objective function and constraint conditions; the day-ahead scheduling of the mobile charging piles is obtained through an aggregator, and then a scheduling strategy is further optimized in a short time scale in a reinforcement learning mode, so that the intelligent scheduling of the mobile charging pile cluster is realized.
In order to achieve the purpose, the invention adopts the following technical scheme:
a mobile charging pile group intelligent scheduling method based on a Sarsa algorithm comprises the following steps:
the aggregator integrates the current state and the real-time electricity price of each mobile charging pile and puts forward the power demand of the mobile charging pile group for participating in auxiliary services;
establishing an aging model for a battery used by the mobile charging pile;
designing a loss cost objective function and constraint conditions of the mobile charging pile battery;
obtaining a day-ahead plan of the mobile charging pile through an aggregator;
and the charging pile automatically moves to an electric automobile charging area and an electric power market auxiliary service area through a scheduling instruction of a Sarsa reinforcement learning algorithm.
In order to optimize the technical scheme, the specific measures adopted further comprise:
further, an accelerated aging test is carried out on the lithium battery of the mobile charging pile, the relation between the capacity attenuation and the cycle number of the power lithium battery is obtained, and the specific result is fitted by adopting the following formula:
in the formula, CBatIs the capacity of the battery, ncFor the number of cycles, KaIs the coefficient to be fitted;
the percentage of lithium battery decay was calculated according to the following formula:
in the formula, CEolFor battery end of life correspondenceCapacity of (C)initIs the initial capacity of the battery;
according to the method, the initial cost of the battery is evenly distributed to various working conditions, and the method is shown as the following formula:
in the formula, EBatFor the cost of use of lithium batteries in specific operating conditions, NBatTo the total number of lithium cells involved, EBat_iniThe initial investment cost of the lithium battery is reduced.
And further, the cycle number of the lithium battery is calculated and obtained by a rain flow counting method according to the current actual operation working condition.
Further, carry out the charge-discharge test to the removal and fill electric pile with the same style lithium cell and obtain the loss cost:
selecting at least 5 different multiplying powers to carry out constant current charging and discharging, respectively obtaining the relationship between the charging and discharging efficiency and the power of the battery through fitting, as shown in the following formula,
ηch＝a·Ich+b
ηdis＝c·Idis+d
in the formula, ηchFor charging efficiency, ηdisFor discharge efficiency, IchFor charging current, IdisA, b, c and d are fitting parameters;
the charging and discharging loss of the mobile charging pile is obtained as follows:
Lch＝r·Ich·Ubat·(1-ηch)·t
in the formula of UbatIs terminal voltage of the battery, LchFor the cost of charge loss, LdisFor the cost of discharge loss, r is the electricity price and t is the duration.
Further, an objective function of minimizing the loss cost of the mobile charging pile group is set as:
min J＝EBat+LBat
LBat＝Lch+Ldis
in the formula, EBatFor aging cost, LBatFor the charge-discharge loss cost, min is the minimum value, and J is the objective function.
Further, the power of the mobile charging pile group meets the following constraints:
wherein M is the number of the mobile charging piles,
power of charging pile for ith, PcommandA power reference value for the secondary service is provided,
the maximum value of power that can be provided for the ith electric pile.
Further, the Sarsa algorithm structure adopted comprises: the method comprises the following steps of intelligently scheduling a decision, an environment, an action, an observation value and a reward; the environment is the dispatching of the mobile charging pile group through an aggregator, the Sarsa algorithm observes corresponding results from the environment through the specific action of dispatching the mobile charging piles, rewards are calculated by taking profit as a target, and the optimal dispatching method is the maximum profit obtained through an intelligent dispatching method.
Further, the unified scheduling step of the reinforcement learning mobile charging pile using the Sarsa algorithm is as follows:
step1. Arbitrary initialization Q Table Q(s)t，at) The Q table is a table for obtaining rewards when different actions are selected, meanwhile, the learning rate α and the discount factor gamma are initialized, the value of α is a number smaller than 1, the value of gamma is between 0 and 1, and a is contained1And a2For example, the Q table of two actions, s1 and s2, can be expressed as follows, and each row in the table records the reward obtained by executing the corresponding action in the current state.
Step2. randomly selecting a scheduling method a from the Q table by adopting a-greedy modetObservation of System revenue RtAnd the next state Si，t；
Step3. update the Q table as follows:
Q(st，at)＝Q(st，at)+α·[Rt+γ·Q(s′t，a′t)-Q(st，at)]
step4. repeating the steps from Step2 to Step3 until the state Si，tTo the target desired value StAnd ending the algorithm.
Further, when the mobile charging pile charges the electric automobile or is in a standby state, the mobile charging pile is in an operation mode; outside the operation mode, the mobile charging pile is in an idle mode and can be used for participating in electric power market auxiliary service.
Further, the mobile charging pile is a charging device capable of moving autonomously, and comprises an energy storage unit, a mobile device and a power electronic device.
The invention has the beneficial effects that: the method provides that the minimum group loss cost of the mobile charging piles is obtained through the Sarsa reinforcement learning algorithm. The model and the corresponding constraint conditions of the whole dispatching system are designed, the whole dispatching system has learning capacity through the use of reinforcement learning, actual application is driven to be gradually optimized through data continuously, and excessive dependence on an accurate physical model is avoided. The invention ensures that the mobile charging pile actively participates in the electric power market auxiliary service and gains profit from the electric power market auxiliary service on the premise of meeting the daily charging service of the mobile charging pile, and can realize the win-win mode of a regional power grid and a mobile charging pile operator.
Drawings
Fig. 1 is a schematic diagram illustrating the steps of the intelligent scheduling method of the present invention.
Fig. 2 is a schematic diagram of a main structure of a mobile charging pile according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a system for participating in the electric power market auxiliary service by the mobile charging pile in one embodiment of the present invention.
FIG. 4 is a schematic diagram of the main structure of reinforcement learning according to an embodiment of the present invention.
Detailed Description
The present invention will now be described in further detail with reference to the accompanying drawings.
It should be noted that the terms "upper", "lower", "left", "right", "front", "back", etc. used in the present invention are for clarity of description only, and are not intended to limit the scope of the present invention, and the relative relationship between the terms and the terms is not limited by the technical contents of the essential changes.
According to the method, the problem of optimizing and scheduling the mobile charging pile cluster in a short time scale is solved by an artificial intelligence field reinforcement learning algorithm Sarsa in a data-driven mode in view of the fact that the situation in the day is complex and a real-time physical model is difficult to accurately establish according to the day-ahead scheduling planning of an aggregator platform, and therefore intelligent scheduling of the mobile charging piles participating in electric power market auxiliary services is achieved.
The invention relates to a mobile charging pile which is an autonomous mobile charging device composed of an energy storage unit, a mobile device, a power electronic device and the like. The mobile charging pile can automatically move to a new energy vehicle to complete charging for responding the demand, and also can automatically move to an auxiliary service area to participate in the electric power market auxiliary service.
The intelligent scheduling method and the intelligent scheduling system realize intelligent scheduling of the mobile charging pile group in a way of an aggregator (a cloud platform in charge of mobile charging pile operation). The power selling company clears and distributes dynamic electricity prices according to the operation condition of a regional power grid, and the cloud aggregator integrates the current state and the real-time electricity prices of the mobile charging piles, and puts forward the power requirements of the mobile charging pile groups for participating in auxiliary services. On the basis, the mobile charging pile group automatically moves to an electric automobile charging area and an electric power market auxiliary service area through a scheduling instruction based on the Sarsa reinforcement learning algorithm. The main objective of the invention is to achieve minimization of the loss cost of the mobile charging pile group.
As shown in fig. 1-4, the present invention comprises the steps of: the aggregator integrates the current state and the real-time electricity price of each mobile charging pile and puts forward the power demand of the mobile charging pile group for participating in auxiliary services;
establishing an aging model for a battery used by the mobile charging pile;
1) the aging cost of the mobile charging pile battery. In certain mobile charging pile applications, the lithium battery used has been determined. In order to obtain an aging model closer to the actual attenuation characteristic of the power lithium battery, the embodiment of the invention directly adopts an accelerated aging test on the same type of lithium battery of the mobile charging pile, and the used working condition provides a circulating working condition of auxiliary service for the mobile charging pile. According to the long-period aging test result, the relation between the capacity attenuation and the cycle number of the power lithium battery can be obtained, and the specific result can be fitted by adopting the following formula:
wherein, CBatIs the capacity of the battery, ncFor the number of cycles, KaAre the coefficients to be fitted.
The cycle number of the lithium battery can be obtained by calculation through a rain flow counting method according to the current actual operation working condition. The method for counting the DoD (Depth of Discharge) by using a rain flow counting method specifically comprises the following steps: (1) setting an initial value SOC value as a maximum value; (2) the rain flows downwards from the inner side of each SOC peak value in sequence and falls down at the next SOC peak value until an opposite SOC peak value larger than the starting point of the opposite SOC peak value is obtained; (3) when the rain flow meets the rain flow under the upper layer flow, the rain flow is stopped immediately; the range of change in DOD is defined by the difference between the start and end points of each stream of rain obtained.
On the basis of this, useRelationship n between DoD and maximum number of charge-discharge cycles in battery manualcF (DOD) is obtainedc. And f, (DOD) can be obtained by offline fitting after actual aging test is carried out on the lithium battery.
From the calculated current battery capacity CBatThe percentage of lithium battery decay can be calculated according to the following formula:
in the formula, CBatAs capacity of the current battery, CEolThe corresponding capacity at the end of the battery life, CinitIs the initial capacity of the battery.
On the basis of obtaining the percentage of attenuation of the lithium battery along with the working conditions, the initial test cost of the battery can be evenly distributed to all the working conditions, as shown in the following formula:
in the formula, EBatFor the cost of use of lithium batteries in specific operating conditions, NBatTo the total number of lithium cells involved, EBat_iniThe initial investment cost of the lithium battery is reduced.
2) The charge and discharge loss cost of the mobile charging pile battery. In order to obtain the charge-discharge efficiency which is closer to the actual power lithium battery for the mobile charging pile, the mobile charging pile is subjected to charge-discharge test by the same type of lithium battery. Selecting at least 5 different multiplying powers to carry out constant current charging and discharging, respectively obtaining the relationship between the charging and discharging efficiency and the power of the battery through fitting, as shown in the following formula,
ηch＝a·Ich+b (4)
ηdis＝c·Idis+d (5)
in the formula, ηchFor charging efficiency, ηdisFor discharge efficiency, IchFor charging current, IdisFor the discharge current, a, b, c, d are fitting parameters.
Therefore, the charging and discharging losses of the mobile charging pile are as follows:
Lch＝r·Ich·Ubat·(1-ηch)·t (6)
in the formula of UbatIs terminal voltage of the battery, LchFor the cost of charge loss, LdisFor the cost of discharge loss, r is the electricity price and t is the duration.
3) Unified scheduling model of mobile charging pile cluster. The objective function of minimizing the loss cost of the mobile charging pile group is set as follows:
min J＝EBat+LBat(8)
in the formula, EBatFor aging cost, LBatFor charge and discharge losses the cost can be obtained by adding up the charge and discharge losses, i.e. LBat＝Lch+LdisMin is the minimum value, and J is the objective function.
In order to meet the requirement of participating in the electric power market auxiliary service, the power of the mobile charging pile group should meet the following constraints:
wherein M is the number of the mobile charging piles,
power of charging pile for ith, PcommandA power reference value for the secondary service is provided,
the maximum value of power that can be provided for the ith electric pile.
4) And intelligently scheduling the mobile charging pile cluster. The invention is based on reinforcement learning SarsaThe day-to-day real-time scheduling of the mobile charging piles is achieved. Compared with the traditional optimal scheduling algorithm based on an accurate physical model, the optimal scheduling result can be obtained through multiple iterations by using reinforcement learning and directly driven by data. Due to the fact that multiple uncertain factors exist in the scheduling process of the short time scale, the optimal scheduling strategy is directly obtained from data by using reinforcement learning, and deviation caused by model uncertainty in the traditional method can be avoided. The main structure of reinforcement learning used is shown in fig. 4, and includes: the method comprises five elements of intelligent scheduling decision, environment, action, observation value and reward. The environment related to the invention is the dispatching of the mobile charging pile group by the aggregator, and the Sarsa algorithm dispatches the specific action a of the mobile charging pile by dispatchingtObserving the corresponding result Q from the environmenttCalculating a reward R with a profit as a goaltThe optimal scheduling strategy is to obtain the maximum benefit through an intelligent scheduling method.
Defining reinforcement learning reward RtFor the defined objective function (8), state StIn order to meet the charging and discharging power required by the auxiliary service of the power market in the future planning, the unified scheduling step for the mobile charging pile in a short time scale by using reinforcement learning of the Sarsa algorithm is as follows:
step1. Arbitrary initialization Q Table Q(s)t，at) The Q table is a table for rewarding when different actions are selected, and meanwhile, the learning rate α and the discount factor gamma are initialized, usually α is a number smaller than 1, and gamma is between 0 and 11And a2For example, the Q table of two actions, s1 and s2, can be expressed as follows, and each row in the table records the reward obtained by executing the corresponding action in the current state.
Step2, randomly selecting a scheduling strategy a from the Q table by adopting a-greedy modetObservation of System revenue RtAnd the next state Si，t；
Step3. update the Q table as follows:
Q(st，at)＝Q(st，at)+α·[Rt+γ·Q(s′t，a′t)-Q(st，at)](11)
step4. repeating the steps from Step2 to Step3 until the state Si，tTo the target desired value StAnd ending the algorithm.
Through the reinforcement learning algorithm based on the Sarsa, the intelligent scheduling optimization result about the mobile charging pile group within 30 minutes can be obtained, and the real-time scheduling result in practical application is subject to the result of the Sarsa algorithm.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.
Claims (10)
1. A mobile charging pile group intelligent scheduling method based on a Sarsa algorithm is characterized by comprising the following steps:
the aggregator integrates the current state and the real-time electricity price of each mobile charging pile and puts forward the power demand of the mobile charging pile group for participating in auxiliary services;
establishing an aging model for a battery used by the mobile charging pile;
designing a loss cost objective function and constraint conditions of the mobile charging pile battery;
acquiring the day-ahead scheduling of the mobile charging pile through an aggregator;
and the charging pile automatically moves to an electric automobile charging area and an electric power market auxiliary service area through a scheduling instruction of a Sarsa reinforcement learning algorithm.
2. The method for intelligent scheduling of mobile charging pile groups according to claim 1, wherein accelerated aging tests are performed on lithium batteries of the mobile charging piles to obtain the relationship between the capacity attenuation and the cycle number of the power lithium batteries, and the specific results are fitted by the following formula:
in the formula, CBatIs the capacity of the battery, ncIs the cycle number, a is the coefficient to be fitted;
the percentage of lithium battery decay was calculated according to the following formula:
in the formula, CEolThe corresponding capacity at the end of the battery life, CinitIs the initial capacity of the battery;
according to the method, the initial cost of the battery is evenly distributed to various working conditions, and the method is shown as the following formula:
in the formula, EBatFor the cost of use of lithium batteries in specific operating conditions, NBatTo the total number of lithium cells involved, EBat_iniThe initial investment cost of the lithium battery is reduced.
3. The intelligent scheduling method of mobile charging pile groups according to claim 2, wherein the cycle number of the lithium battery is calculated and obtained by a rain flow counting method according to the current actual operation condition.
4. The method for intelligent scheduling of mobile charging pile groups according to claim 1, wherein the loss cost is obtained by performing charging and discharging tests on the mobile charging piles by using the same type of lithium battery:
selecting at least 5 different multiplying powers to carry out constant current charging and discharging, respectively obtaining the relationship between the charging and discharging efficiency and the power of the battery through fitting, as shown in the following formula,
ηch＝a·Ich+b
ηdis＝c·Idis+d
in the formula, ηchFor charging efficiency, ηdisFor discharge efficiency, IchFor charging current, IdisA, b, c and d are fitting parameters;
the charging and discharging loss of the mobile charging pile is obtained as follows:
Lch＝r·Ich·Ubat·(1-ηch)·t
in the formula of UbatIs terminal voltage of the battery, LchFor the cost of charge loss, LdisFor the cost of discharge loss, r is the electricity price and t is the duration.
5. The method for intelligent scheduling of a mobile charging pile group according to claim 1, wherein an objective function of minimizing loss cost of the mobile charging pile group is set as:
min J＝EBat+LBat
LBat＝Lch+Ldis
in the formula, EBatFor aging cost, LBatFor the charge-discharge loss cost, min is the minimum value, and J is the objective function.
6. The method of claim 1, wherein the power of the mobile charging pile group meets the following constraints:
in which M is a transitionThe number of the movable charging piles is increased,
power of charging pile for ith, PcommandA power reference value for the secondary service is provided,
the maximum value of power that can be provided for the ith electric pile.
7. The method for intelligent scheduling of mobile charging pile groups according to claim 1, wherein the Sarsa algorithm structure adopted comprises: the method comprises the following steps of intelligently scheduling a decision, an environment, an action, an observation value and a reward; the environment is the dispatching of the mobile charging pile group through an aggregator, the Sarsa algorithm observes corresponding results from the environment through the specific action of dispatching the mobile charging piles, rewards are calculated by taking profit as a target, and the optimal dispatching method is the maximum profit obtained through an intelligent dispatching method.
8. The method for intelligent scheduling of mobile charging pile groups according to claim 7, wherein the step of unified scheduling of reinforcement learning mobile charging piles using Sarsa algorithm is as follows:
step1. Arbitrary initialization Q Table Q(s)t，at) The Q table is a table for obtaining rewards when different actions are selected, meanwhile, the learning rate α and the discount factor gamma are initialized, the value of α is a number smaller than 1, the value of gamma is between 0 and 1, Step2, a scheduling method a is randomly selected from the Q table in a-greedy modetObservation of System revenue RtAnd the next state Si，t；
Step3. update the Q table as follows:
Q(st，at)＝Q(st，at)+α·[Rt+γ·Q(s′t，a′t)-Q(st，at)]
step4. repeating the steps from Step2 to Step3 until the state Si，tTo the target desired value StAnd ending the algorithm.
9. The method for intelligent scheduling of mobile charging pile groups according to claim 1, wherein when the mobile charging piles charge the electric vehicles or are in a standby state, the mobile charging piles are in an operation mode; outside the operation mode, the mobile charging pile is in an idle mode and can be used for participating in electric power market auxiliary service.
10. The method for intelligent scheduling of mobile charging pile groups according to claim 1, wherein the mobile charging piles are charging devices capable of moving autonomously, and comprise energy storage units, mobile devices and power electronic devices.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010607160.8A CN111738611B (en) | 2020-06-29 | 2020-06-29 | Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010607160.8A CN111738611B (en) | 2020-06-29 | 2020-06-29 | Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111738611A true CN111738611A (en) | 2020-10-02 |
| CN111738611B CN111738611B (en) | 2024-04-26 |
Family
ID=72652775
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010607160.8A Active CN111738611B (en) | 2020-06-29 | 2020-06-29 | Intelligent scheduling method for mobile charging pile group based on Sarsa algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111738611B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112590598A (en) * | 2020-12-11 | 2021-04-02 | 合肥工业大学 | Optimal configuration method and system for mobile charging vehicle |
| CN116629413A (en) * | 2023-05-10 | 2023-08-22 | 金陵科技学院 | Global optimal automatic deployment method for mobile charging device based on random forest and Dijkstra |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105046371A (en) * | 2015-08-19 | 2015-11-11 | 东南大学 | Electric vehicle charge-discharge scheduling method based on demand side bidding |
| KR20160030643A (en) * | 2014-09-11 | 2016-03-21 | 한밭대학교 산학협력단 | Apparatus and method for real-time estimation of battery state-of-health |
| CN106469354A (en) * | 2016-09-09 | 2017-03-01 | 中国电力科学研究院 | A kind of user's request response participatory approaches under Load aggregation quotient module formula |
| CN207560060U (en) * | 2017-12-26 | 2018-06-29 | 数源科技股份有限公司 | Intelligent mobile charging pile group system |
| EP3386059A1 (en) * | 2017-04-06 | 2018-10-10 | ABB Schweiz AG | Method and system for providing ancillary grid services |
| CN109510250A (en) * | 2018-12-19 | 2019-03-22 | 洁电（北京）储能科技有限公司 | Charging station, charging pile system and control method with power grid ancillary service function |
| CN109816171A (en) * | 2019-01-28 | 2019-05-28 | 三峡大学 | A kind of electric car region microgrid group's bilayer distributed optimization dispatching method based on fuzzy Spot Price |
| CN109910671A (en) * | 2019-03-11 | 2019-06-21 | 三峡大学 | Electric car V2G control method based on intelligent charging spot |
| CN110210709A (en) * | 2019-04-30 | 2019-09-06 | 太原理工大学 | A kind of frequency modulation instruction dispatching method counted and energy-accumulating power station life time decay and capacity equilibrium are bidded |
| CN110281809A (en) * | 2019-05-28 | 2019-09-27 | 北京航盛新能科技有限公司 | A kind of electric car V2G control system based on particle swarm optimization algorithm |
| CN111016725A (en) * | 2019-12-31 | 2020-04-17 | 西安交通大学 | Mode selection control method of energy storage type charging pile |
| CN111106612A (en) * | 2018-10-25 | 2020-05-05 | 贵州电网有限责任公司电力科学研究院 | Energy storage type charging pile participating power grid demand side response combined operation optimization model and solving algorithm |
| CN210502331U (en) * | 2019-07-17 | 2020-05-12 | 深圳市瑞能实业股份有限公司 | Energy storage charging pile system |
-
2020
- 2020-06-29 CN CN202010607160.8A patent/CN111738611B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160030643A (en) * | 2014-09-11 | 2016-03-21 | 한밭대학교 산학협력단 | Apparatus and method for real-time estimation of battery state-of-health |
| CN105046371A (en) * | 2015-08-19 | 2015-11-11 | 东南大学 | Electric vehicle charge-discharge scheduling method based on demand side bidding |
| CN106469354A (en) * | 2016-09-09 | 2017-03-01 | 中国电力科学研究院 | A kind of user's request response participatory approaches under Load aggregation quotient module formula |
| EP3386059A1 (en) * | 2017-04-06 | 2018-10-10 | ABB Schweiz AG | Method and system for providing ancillary grid services |
| CN207560060U (en) * | 2017-12-26 | 2018-06-29 | 数源科技股份有限公司 | Intelligent mobile charging pile group system |
| CN111106612A (en) * | 2018-10-25 | 2020-05-05 | 贵州电网有限责任公司电力科学研究院 | Energy storage type charging pile participating power grid demand side response combined operation optimization model and solving algorithm |
| CN109510250A (en) * | 2018-12-19 | 2019-03-22 | 洁电（北京）储能科技有限公司 | Charging station, charging pile system and control method with power grid ancillary service function |
| CN109816171A (en) * | 2019-01-28 | 2019-05-28 | 三峡大学 | A kind of electric car region microgrid group's bilayer distributed optimization dispatching method based on fuzzy Spot Price |
| CN109910671A (en) * | 2019-03-11 | 2019-06-21 | 三峡大学 | Electric car V2G control method based on intelligent charging spot |
| CN110210709A (en) * | 2019-04-30 | 2019-09-06 | 太原理工大学 | A kind of frequency modulation instruction dispatching method counted and energy-accumulating power station life time decay and capacity equilibrium are bidded |
| CN110281809A (en) * | 2019-05-28 | 2019-09-27 | 北京航盛新能科技有限公司 | A kind of electric car V2G control system based on particle swarm optimization algorithm |
| CN210502331U (en) * | 2019-07-17 | 2020-05-12 | 深圳市瑞能实业股份有限公司 | Energy storage charging pile system |
| CN111016725A (en) * | 2019-12-31 | 2020-04-17 | 西安交通大学 | Mode selection control method of energy storage type charging pile |
Non-Patent Citations (10)
| Title |
|---|
| YONGSHENG CAO 等: "Joint Optimization of Delay-Tolerant Autonomous Electric Vehicles Charge Scheduling and Station Bateery Degradation", 《IEEE INTERNET OF THINGS JOURAL》, vol. 7, no. 9, pages 8590 - 859 * |
| ZHENG WANG 等: "A load Scheduling Strategy for Electric Vehicles Charging System", 《2014 IEEEEIGHTH INTERNATIONAL CONFERENCE ON SOFTWARE SECURITY AND RELIABILITY-COMPANION(SERE-C 2014)》, pages 218 - 222 * |
| 吕凯: "智能电网中多储能装置的学习优化控制", 中国优秀硕士学位论文全文数据库, no. 2016, pages 042 - 591 * |
| 姜伊: "考虑寿命的电动汽车动力电池参与电网调频策略研究", 中国优秀硕士学位论文全文数据库, no. 2017, pages 035 - 786 * |
| 曹括: "V2G充电桩与智能微电网的并网技术研究", 中国优秀硕士学位论文全文数据库, no. 2019, pages 034 - 247 * |
| 林立乾 等: "面向电力市场的分布式储能聚合参与电网调峰", 储能科学与技术, no. 201902, pages 276 - 283 * |
| 王宁: "锂离子电池寿命预测研究", 中国优秀硕士学位论文全文数据库, no. 2019, pages 042 - 1189 * |
| 裴荣娜: "基于V2G技术的电动汽车充电桩的充放电控制策略研究", 中国优秀硕士学位论文全文数据库, no. 2017, pages 034 - 1159 * |
| 赵亚杰: "电动汽车聚合服务商参与电网辅助服务的调控策略研究", 中国优秀硕士学位论文全文数据库, no. 2020, pages 042 - 86 * |
| 陈浩宇: "考虑电动汽车的并网型微电网优化协调调度", 中国优秀硕士学位论文全文数据库, no. 2019, pages 035 - 360 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112590598A (en) * | 2020-12-11 | 2021-04-02 | 合肥工业大学 | Optimal configuration method and system for mobile charging vehicle |
| CN112590598B (en) * | 2020-12-11 | 2023-11-07 | 合肥工业大学 | Optimal configuration method and system for mobile charging vehicle |
| CN116629413A (en) * | 2023-05-10 | 2023-08-22 | 金陵科技学院 | Global optimal automatic deployment method for mobile charging device based on random forest and Dijkstra |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111738611B (en) | 2024-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103969585B (en) | Assess method and apparatus, related system and the vehicle of the behaviour in service of battery | |
| CN113176511B (en) | Energy storage charging and discharging optimization method and system considering health state | |
| CN109193721B (en) | Electric vehicle charging and discharging strategy optimization method based on reinforcement learning | |
| CN105226689A (en) | Consider polymorphic type energy-storage system energy management method and the system of operation and maintenance | |
| CN111833205A (en) | Mobile charging pile group intelligent scheduling method in big data scene | |
| CN104698382A (en) | Method for predicting the SOC and SOH of battery pack | |
| CN111244991A (en) | Energy storage power station operation control method and system considering battery life | |
| CN110415016A (en) | A kind of charging pricing practice strategy based on optimization charge and discharge strategy | |
| CN113644651A (en) | Energy storage configuration optimization method under electricity price bidding scene | |
| CN109802412A (en) | The Optimal Configuration Method of user side load aggregation quotient's stored energy capacitance | |
| CN107045104A (en) | A kind of On-line Estimation method of lithium titanate battery capacity | |
| CN108732499A (en) | A kind of method and system of detection cycle life of lithium ion battery | |
| CN116093466A (en) | Ordered charging method, ordered charging device, computer readable storage medium and computer equipment | |
| CN106451505A (en) | Output regulation method and device for distributed energy storage system | |
| CN110021947B (en) | Distributed energy storage power system operation optimization method based on reinforcement learning | |
| CN111738611A (en) | Mobile charging pile group intelligent scheduling method based on Sarsa algorithm | |
| CN107776433A (en) | A kind of discharge and recharge optimal control method of electric automobile group | |
| CN109523079B (en) | Optimization method for combined auxiliary peak shaving by utilizing electric automobile and heat accumulating electric boiler | |
| CN111724080A (en) | Mobile charging pile group scheduling method considering battery pack health state balance | |
| CN111680829A (en) | Distributed energy storage cluster scheduling method considering charge and discharge cost | |
| CN113495212A (en) | Method for estimating maintenance regeneration times and cycle life of maintainable regenerative battery | |
| CN111525597A (en) | Method for optimizing double-battery imbalance state in wind storage combined system | |
| Ye et al. | Optimal control strategy for plug-in electric vehicles based on reinforcement learning in distribution networks | |
| CN106230010B (en) | Capacity optimization configuration method and system for hundred megawatt battery energy storage system | |
| CN106253356A (en) | Alternating current-direct current mixing microgrid Real-time Economic Dispatch method based on energy storage electricity value assessment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |