CN111833205A - An intelligent scheduling method for mobile charging pile groups in big data scenarios - Google Patents

Abstract

The invention discloses an intelligent scheduling method for a mobile charging pile group in a big data scenario, comprising the following steps: establishing a scheduling model of a mobile charging pile cluster; solving the scheduling model through constraints and a DQN algorithm to obtain an intelligent scheduling strategy ; Schedule mobile charging piles according to the intelligent scheduling strategy. Through the dispatching model of the mobile charging pile cluster, all mobile charging piles in the area are uniformly dispatched to participate in the auxiliary services of the power market, which increases the profit method of mobile charging piles, and also provides services for the power grid, which alleviates the problems of regional power system frequency regulation and new energy consumption. .

Description

An intelligent scheduling method for mobile charging pile groups in big data scenarios

technical field

The invention belongs to the application field of charging devices, and relates to an intelligent scheduling method for mobile charging pile groups in a big data scenario.

Background technique

The use of mobile charging piles to achieve electric vehicle charging has high mobility and solves the problem of low utilization of fixed charging piles. In addition, with the explosive growth of new energy vehicles, large-scale retired power lithium batteries will be produced. Mobile charging piles have also become an important way to give full play to the residual value of retired power lithium batteries. In the form of aggregator cloud platform, it is of great significance to operate idle mobile charging piles to participate in auxiliary services in the power market for maximizing the value of mobile charging piles.

To participate in the auxiliary services of the power market, a large number of mobile charging piles must be combined to form a certain scale, so as to effectively provide services for the regional power grid. At this time, it is necessary for the aggregator to collect the status information of each mobile charging pile, and through the cloud platform calculation, uniformly dispatch all the charging piles to participate in the electric auxiliary service to maximize the benefits. If there are a large number of mobile charging piles in a specific area, due to the existence of various uncertain factors, the physical model of the system will be difficult to establish accurately. The optimal scheduling methods based on deterministic models, such as Model Predictive Control (MPC), It will be difficult to effectively implement system scheduling. However, traditional reinforcement learning algorithms such as Q-learning and Sarsa cannot deal with continuous state variables, which will restrict the accuracy of the scheduling strategy.

SUMMARY OF THE INVENTION

In view of the deficiencies of the prior art, the purpose of the present invention is to provide an intelligent scheduling method for a mobile charging pile group in a big data scenario, so as to solve the problem of difficult system scheduling in the prior art.

In order to solve the above-mentioned technical problems, the technical scheme adopted in the present invention is:

An intelligent scheduling method for mobile charging pile groups in a big data scenario, comprising the following steps:

Establish a scheduling model for mobile charging pile clusters;

Solve the scheduling model through constraints and the DQN algorithm to obtain an intelligent scheduling strategy;

The mobile charging pile is scheduled according to the intelligent scheduling strategy.

Further, the cost function of the scheduling model includes:

min J=B _agg +E _Bat (7)

Among them, J is the cost function of mobile charging pile cluster scheduling, E _Bat is the use cost of lithium batteries in a specific working condition, and _Bagg is the overall income of the aggregator.

Further, the overall revenue of the aggregator is:

B _agg =B _c ·w _c +B _e ·we ₍ 1)

In the formula, B _agg is the total revenue of the aggregator; B _c is the benefit obtained by the aggregator actively participating in the auxiliary services of the electricity market; w _c is the sharing coefficient of the aggregator actively participating in the auxiliary services of the electricity market; B _e is the gain from energy arbitrage Profit; we _e is the profit sharing coefficient of the aggregator in the way of energy arbitrage.

Further, the benefits obtained by the aggregator actively participating in the auxiliary services of the electricity market are:

Among them, rp and _rv are the compensation electricity prices for peak and valley, respectively, and _{P p and} P _v are the total power that the aggregator can provide during peak and valley, respectively;

The profit obtained from the energy arbitrage is:

Among them, Qi _,t is the charge and discharge power available in the control period.

Further, the use cost of the lithium battery in the specific working condition is:

为锂电池衰减的百分比。Among them, E _Bat is the use cost of lithium batteries in a specific working condition, N _Bat is the total number of lithium batteries involved, E _{Bat_ini} is the initial test investment cost of lithium batteries,

It is the percentage of lithium battery attenuation.

Further, the decay percentage of the lithium battery is:

Among them, C _Bat is the current capacity of the battery, C _Eol is the capacity corresponding to the end of the battery life, and C _init is the initial capacity of the battery.

Further, the capacity of the current battery is:

C _Bat = a·n _c ^b +c (4)

Among them, C _Bat is the current capacity of the battery, n _c is the number of cycles, a is the coefficient of the power function, b is the number of times of the power function, and c is the offset.

Further, the constraints include movement range constraints:

为当前移动充电桩的移动距离，L_max为移动充电桩的最大许可移动距离；in,

is the moving distance of the current mobile charging pile, and L _max is the maximum allowable moving distance of the mobile charging pile;

Constraints on the number of mobile charging piles:

为参与聚合商统一调度的移动充电桩数目，N_max为可参与统一调度的移动充电桩最大许可数目；in,

is the number of mobile charging piles that participate in the unified scheduling of the aggregator, and N _max is the maximum permitted number of mobile charging piles that can participate in the unified scheduling;

Power constraints of mobile charging piles:

和

分别为电池储能单元许可充、放电功率的最大值；In the formula, P _ch and P _dis are the allowable charge and discharge power of the battery energy storage unit, respectively;

and

are the maximum allowable charge and discharge power of the battery energy storage unit respectively;

Capacity constraints of mobile charging piles:

与

分别为电池储能单元荷电状态的上下限值。in,

and

are the upper and lower limits of the state of charge of the battery energy storage unit, respectively.

Further, the DQN algorithm includes:

与折扣因子；Initialize observations Q(s _t , at _t ),

and discount factor;

Select the scheduling strategy a _t with ε as the probability, and observe the system's revenue r _t and state s _t+1 ;

Store (s _t , at , r _t , s _{t +1} ) in the playback memory unit D;

Randomly extract an appropriate amount of learning experience (s _t , at , r _t , s _{t +1} ) from D to train the target neural network;

Using the gradient descent method, the current neural network is trained by minimizing the loss function;

Copy the current neural network parameters to the target neural network every N time windows;

结束算法。Repeat the above steps until the state s _t reaches the target expected value

End the algorithm.

Further, the neural network training output result is:

The minimized loss function is:

(y _j -Q(s _j ,a _j |θ)) ² ,

Among them, r _j is the jth reward, γ is the discount factor, Q is the observation value, θ is the parameter of the neural network, s _j is the jth state, and a _j is the jth action.

A mobile charging pile group intelligent dispatching system in a big data scenario, the system includes:

Scheduling model module: used to establish a scheduling model for mobile charging pile clusters;

Solving module: used to solve the scheduling model through constraints and DQN algorithm to obtain an intelligent scheduling strategy;

Scheduling module: used to schedule the mobile charging pile according to the intelligent scheduling strategy.

A mobile charging pile group intelligent dispatching system in a big data scenario, the system includes a processor and a storage medium;

the storage medium is used for storing instructions;

The processor is adapted to operate in accordance with the instructions to perform steps in accordance with the method described above.

A computer-readable storage medium having a computer program stored thereon, the program implementing the steps of the above-described method when executed by a processor.

Compared with the prior art, the beneficial effects achieved by the present invention are:

Through the dispatching model of the mobile charging pile cluster, all mobile charging piles in the area are uniformly dispatched to participate in the auxiliary services of the power market, which increases the profit method of mobile charging piles, and also provides services for the power grid, which alleviates the problems of regional power system frequency regulation and new energy consumption. , is a win-win operation mode; the invention provides a group intelligent scheduling strategy of reinforcement learning DQN for mobile charging piles in big data application scenarios, which can help aggregator cloud platforms make real-time scheduling decisions and obtain maximum benefits.

Description of drawings

1 is a system structure of a cloud platform system for group scheduling of mobile charging piles based on aggregators in an embodiment of the present invention;

FIG. 2 is the overall framework of the reinforcement learning DQN algorithm in an embodiment of the present invention.

Detailed ways

The specific implementations of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific implementation manners described herein are only used to illustrate and explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention.

The present invention firstly establishes a model for scheduling mobile charging pile groups in an aggregator mode, and designs corresponding constraints; on this basis, a cost model for mobile charging pile scheduling is established; The DQN algorithm is used to obtain the intelligent scheduling strategy of the mobile charging pile group.

Figure 1 shows the system structure of dispatching mobile charging piles in a certain area to participate in the auxiliary services of the electricity market in the way of aggregators. It mainly includes important components such as mobile charging piles, aggregator cloud platforms and regional power grids. The mobile charging pile includes: the energy storage unit is used to store and release electric energy, and the autonomous mobile unit can move to the target vehicle to provide charging services for it, and can also follow the dispatching instructions of the cloud platform to participate in the auxiliary services of the power market. In addition, the mobile charging pile must also include a wireless communication device for data exchange with the cloud platform. The mobile charging pile can work in the service state and the idle state. The service state is used to charge electric vehicles, and the rest of the time in the idle state can be used for energy exchange with the power grid to provide auxiliary services for the regional power grid. Aggregators use the cloud platform to uniformly schedule each mobile charging pile. The cloud platform needs to collect the real-time status of each mobile charging pile, including: state of charge, power status, idle status, and scheduling costs. After that, according to the electricity price information of the regional power grid, etc., the DQN is used to obtain the optimal scheduling strategy to maximize the benefits of the mobile charging pile group. The regional power grid determines the regional electricity price based on the comprehensive judgment of the local load and the consumption of new energy.

1) Revenue calculation of the aggregator cloud platform. The income of the aggregator mainly includes the income of participating in the auxiliary services of the electricity market, and realizing the energy arbitrage in the way of "high power generation and low storage" according to the real-time electricity price of the regional power grid. Therefore, the aggregator's revenue can be expressed as:

B _agg =B _c ·w _c +B _e ·we ₍ 1)

In the formula, B _agg is the total revenue of the aggregator; B _c is the benefit obtained by the aggregator actively participating in the auxiliary services of the electricity market; w _c is the sharing coefficient of the aggregator actively participating in the auxiliary services of the electricity market; B _e is the gain from energy arbitrage Profit; we _e is the profit sharing coefficient of the aggregator in the way of energy arbitrage.

According to the profit method of the aggregator cloud platform, B _c and B _e are respectively defined as follows:

Among them, rp and _rv are the compensated electricity prices for peak and valley, respectively, _{P p and} P _v are the total power that the aggregator can provide during peak and valley, respectively, and Qi _,t is the charge and discharge power that can be provided in the control period.

2) The cost of using mobile charging piles. The battery type selected by the mobile charging pile has been determined, and the present invention will conduct a cycle accelerated aging test through a plurality of lithium batteries of the same type. In the embodiment of the present invention, the actual working condition of ancillary services participating in the power market is taken as the cycle working condition, and data is collected through a long-cycle aging test to obtain the relationship between the battery capacity decline and the number of cycles. The present invention obtains the cycle life empirical model of the lithium battery through the superposition of the power function and the linear function, and through nonlinear least square fitting, as shown in the following formula:

C _Bat = a·n _c ^b +c (4)

Among them, C _Bat is the current capacity of the battery, n _c is the number of cycles, and a, b, and c are the parameters to be fitted.

In actual operation, the number of cycles can be obtained by the rainflow counting method. The rules of the rainflow counting method include: (1) set the initial value to the maximum value; (2) the rainflow flows downward from the inner side of each peak in turn, and falls at the next peak until there is a more opposite one than its starting point. Large peak; (3). When the rain flow encounters the rain flow down from the upper layer, it stops immediately. After the above steps, the number of cycles corresponding to a specific working condition can be calculated.

According to the calculated current battery capacity C _Bat , the percentage of lithium battery decay can be calculated by the following formula:

In the formula, C _Bat is the current capacity of the battery, C _Eol is the corresponding capacity at the end of the battery life, and C _init is the initial capacity of the battery.

On the basis of obtaining the decay percentage of the lithium battery, the initial investment cost of the battery can be amortized into each cycle condition to obtain the battery scheduling cost, as shown in the following formula:

In the formula, E _Bat is the use cost of lithium batteries in a specific working condition, N _Bat is the total number of lithium batteries involved, and E _{Bat_ini} is the initial investment cost of lithium batteries.

3) Scheduling model of mobile charging pile clusters. Based on the above system structure and income analysis, the following objective function can be set for the scheduling of mobile charging pile clusters.

min J=B _agg +E _Bat (7)

The above formula comprehensively considers the benefits of the aggregator cloud platform and the dispatching cost of the mobile charging pile energy storage unit. In addition, the unified scheduling of mobile charging piles also needs to consider the following constraints.

The operating range of mobile charging piles is limited. For each mobile charging pile, it is impossible to move in a large wireless range, and its usual moving range will be limited to a certain range, that is, unified scheduling needs to meet the following conditions:

为当前移动充电桩的移动距离，L_max为移动充电桩的最大许可移动距离。in,

is the moving distance of the current mobile charging pile, and L _max is the maximum allowable moving distance of the mobile charging pile.

The number of mobile charging piles participating in unified scheduling is limited. Due to the constraints of various factors such as site and cable power in practical applications, the number of mobile charging piles participating in unified scheduling has an upper limit. That is, the number of mobile charging piles that can participate in the unified scheduling of the aggregator should meet the following constraints:

为参与聚合商统一调度的移动充电桩数目，N_max为可参与统一调度的移动充电桩最大许可数目。in,

is the number of mobile charging piles that participate in the unified scheduling of the aggregator, and N _max is the maximum permitted number of mobile charging piles that can participate in the unified scheduling.

In addition, the power limit of each mobile charging pile can be expressed as follows:

和

分别为电池储能单元许可充、放电功率的最大值。In the formula,

and

are the maximum allowable charge and discharge power of the battery energy storage unit, respectively.

The capacity limit of the mobile charging pile can be expressed as:

与

分别为电池储能单元荷电状态的上下限值。in,

and

are the upper and lower limits of the state of charge of the battery energy storage unit, respectively.

4) Intelligent scheduling of mobile charging pile clusters. The invention adopts DQN to realize the intelligent scheduling of mobile charging pile groups in the big data scenario by means of reinforcement learning combined with deep learning. First, through the established system model, the optimization problem with constraints is solved by quadratic programming, and the coarse time-scale optimization of day-ahead scheduling is obtained. In this embodiment, in order to realize the day-ahead scheduling on a long-term scale, the dynamic electricity price will be released by the electricity sales company, and formula (7) is used as the objective function, and (8) to (11) are selected as constraints to carry out long-term mobile charging piles. Cluster scheduling.

On this basis, the DQN method is selected to gradually optimize the entire scheduling strategy through reinforcement learning, and a more refined scheduling is performed at 1 hour intervals. The main structure of the DQN algorithm is shown in Figure 2, including: environment, action a, observation value Q, reward r and so on. Reinforcement learning establishes the connection between the current state and the action that should be taken in a data-driven manner by taking actions, interacting with the environment, observing outcomes, and obtaining rewards. Compared with traditional reinforcement learning Q-learning and Sarsa, the DQN used in the present invention can process continuous states due to the use of neural networks.

Define the reward r of the reinforcement learning DQN as the objective function (7), the state s is the charging and discharging capacity required to meet the auxiliary services of the electricity market in the previous planning, and the action a is the specific strategy used in each dispatch, using the DQN algorithm to complete the short-term The unified scheduling steps of time-scale charging piles are as follows:

以及折扣因子γ，γ取值为0～1之间。Step1. Arbitrarily initialize the Q table Q(s _t , at _t ),

and the discount factor γ, where γ ranges from 0 to 1.

Step2. Select the scheduling strategy at _t with ε as the probability, and observe the income r _t and the state s _t+1 of the system;

Step3. Store (s _t , at , r _t , s _{t +1} ) in the playback memory unit D;

Step4. Randomly extract a small amount of learning experience (s _t , at , r _t , s _{t +1} ) from D for training the target neural network, and define the following target neural network training output results:

Step5. Use the gradient descent method to minimize the loss function (y _j -Q(s _j ,a _j |θ)) ² to train the current neural network, where θ is the parameter of the neural network;

Step6. Every N time windows, copy the current neural network parameters to the target neural network.

结束算法。Step7. Repeat steps Step2 to Step6 until the state s _t reaches the target expected value

End the algorithm.

After the above training process of DQN reinforcement learning, an intelligent scheduling strategy for mobile charging pile clusters that maximizes the benefits of the aggregator cloud platform can be obtained, and the optimal scheduling of mobile charging pile groups can be achieved within an hour timescale.

Facing the big data application scenarios after the popularization of mobile charging piles in the future, the present invention proposes to use Deep QNetwork (DQN) to combine deep learning and reinforcement learning, and use a large amount of data to directly drive, and continue to strengthen after training. DQN's mobile charging pile group intelligent scheduling strategy enables idle mobile charging piles to actively participate in the auxiliary services of the electricity market to make profits, effectively improving the economic benefits of operators.

In order to realize the intelligent scheduling of mobile charging pile clusters of a certain scale, the present invention needs an optimal scheduling algorithm that can process large data information including different state quantities. To this end, a DQN-based mobile charging pile group intelligent scheduling method is proposed, which can combine deep learning and reinforcement learning to deal with big data scenarios, process continuous state variables, and obtain a unified platform for a large number of mobile charging piles under the aggregator cloud platform mode. Smart scheduling strategy.

A mobile charging pile group intelligent dispatching system in a big data scenario, the system includes:

Scheduling model module: used to establish a scheduling model for mobile charging pile clusters;

Solving module: used to solve the scheduling model through constraints and DQN algorithm to obtain an intelligent scheduling strategy;

Scheduling module: used to schedule the mobile charging pile according to the intelligent scheduling strategy.

A mobile charging pile group intelligent dispatching system in a big data scenario, the system includes a processor and a storage medium;

the storage medium is used for storing instructions;

The processor is adapted to operate in accordance with the instructions to perform steps in accordance with the method described above.

A computer-readable storage medium having a computer program stored thereon, the program implementing the steps of the above-described method when executed by a processor.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flows of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The above are only examples of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the application for pending approval of the present invention. within the scope of the claims.

Claims (10)

1. A method for intelligent scheduling of mobile charging pile groups under a big data scenario, characterized in that it comprises the following steps: Establish a scheduling model for mobile charging pile clusters; Solve the scheduling model through constraints and the DQN algorithm to obtain an intelligent scheduling strategy; The mobile charging pile is scheduled according to the intelligent scheduling strategy. 2. The method for intelligent scheduling of mobile charging pile groups in a big data scenario according to claim 1, wherein the cost function of the scheduling model comprises: min J=B _agg +E _Bat (7) Among them, J is the cost function of mobile charging pile cluster scheduling, E _Bat is the use cost of lithium batteries in a specific working condition, and _Bagg is the overall income of the aggregator. 3. The method for intelligent scheduling of mobile charging pile groups in a big data scenario according to claim 2, wherein the overall revenue of the aggregator is: B _agg =B _c ·w _c +B _e ·we ₍ 1) In the formula, B _agg is the total revenue of the aggregator; B _c is the benefit obtained by the aggregator actively participating in the auxiliary services of the electricity market; w _c is the sharing coefficient of the aggregator actively participating in the auxiliary services of the electricity market; B _e is the gain from energy arbitrage Profit; we _e is the profit sharing coefficient of the aggregator in the way of energy arbitrage. 4. The method for intelligent scheduling of mobile charging pile groups under a big data scenario according to claim 3, wherein the benefits obtained by the aggregator actively participating in the auxiliary services of the electricity market are:

Among them, rp and _rv are the compensation electricity prices for peak and valley, respectively, and _{P p and} P _v are the total power that the aggregator can provide during peak and valley, respectively; The profit obtained from the energy arbitrage is:

Among them, Qi _,t is the charge and discharge power available in the control period. 5. The method for intelligent scheduling of mobile charging pile groups under a big data scenario according to claim 2, wherein the use cost of the lithium battery in the specific working condition is:

Among them, E _Bat is the use cost of lithium batteries in a specific working condition, N _Bat is the total number of lithium batteries involved, E _{Bat_ini} is the initial test investment cost of lithium batteries,

It is the percentage of lithium battery attenuation. 6 . The method for intelligent scheduling of mobile charging pile groups in a big data scenario according to claim 5 , wherein the decay percentage of the lithium battery is:

Among them, C _Bat is the current capacity of the battery, C _Eol is the capacity corresponding to the end of the battery life, and C _init is the initial capacity of the battery. 7. The intelligent scheduling method for mobile charging pile groups in a big data scenario according to claim 6, wherein the current capacity of the battery is: C _Bat = a·n _c ^b +c (4) Among them, C _Bat is the current capacity of the battery, n _c is the number of cycles, a is the coefficient of the power function, b is the number of times of the power function, and c is the offset. 8 . The method for intelligent scheduling of mobile charging pile groups in a big data scenario according to claim 1 , wherein the constraints include movement range constraints:

in,

is the moving distance of the current mobile charging pile, and L _max is the maximum allowable moving distance of the mobile charging pile; Constraints on the number of mobile charging piles:

in,

is the number of mobile charging piles that participate in the unified scheduling of the aggregator, and N _max is the maximum permitted number of mobile charging piles that can participate in the unified scheduling; Power constraints of mobile charging piles:

In the formula, P _ch and P _dis are the allowable charge and discharge power of the battery energy storage unit, respectively;

and

are the maximum allowable charge and discharge power of the battery energy storage unit respectively; Capacity constraints of mobile charging piles:

in,

and

are the upper and lower limits of the state of charge of the battery energy storage unit, respectively. 9. The intelligent scheduling method for mobile charging pile groups under a big data scenario according to claim 1, wherein the DQN algorithm comprises: Initialize observations Q(s _t , at _t ),

with the discount factor γ; Select the scheduling strategy a _t with ε as the probability, and observe the system's revenue r _t and state s _t+1 ; Store (s _t , at , r _t , s _{t +1} ) in the playback memory unit D; Randomly extract an appropriate amount of learning experience (s _t , at , r _t , s _{t +1} ) from D to train the target neural network; Using the gradient descent method, the current neural network is trained by minimizing the loss function; Copy the current neural network parameters to the target neural network every N time windows; Repeat the above steps until the state s _t reaches the target expected value

End the algorithm. 10. The method for intelligent scheduling of mobile charging pile groups in a big data scenario according to claim 9, wherein the neural network training output result is:

The minimized loss function is: (y _j -Q(s _j ,a _j |θ)) ² , Among them, r _j is the jth reward, γ is the discount factor, Q is the observation value, θ is the parameter of the neural network, s _j is the jth state, and a _j is the jth action.

Images