CN114444802A - Electric vehicle charging guide optimization method based on graph neural network reinforcement learning - Google Patents

Electric vehicle charging guide optimization method based on graph neural network reinforcement learning Download PDF

Info

Publication number
CN114444802A
CN114444802A CN202210109887.2A CN202210109887A CN114444802A CN 114444802 A CN114444802 A CN 114444802A CN 202210109887 A CN202210109887 A CN 202210109887A CN 114444802 A CN114444802 A CN 114444802A
Authority
CN
China
Prior art keywords
node
charging
electric vehicle
neural network
reinforcement learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210109887.2A
Other languages
Chinese (zh)
Other versions
CN114444802B (en
Inventor
江昌旭
卢玥君
林铮
邵振国
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fuzhou University
Original Assignee
Fuzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fuzhou University filed Critical Fuzhou University
Priority to CN202210109887.2A priority Critical patent/CN114444802B/en
Publication of CN114444802A publication Critical patent/CN114444802A/en
Application granted granted Critical
Publication of CN114444802B publication Critical patent/CN114444802B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0283—Price estimation or determination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Energy or water supply
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Economics (AREA)
  • Theoretical Computer Science (AREA)
  • Strategic Management (AREA)
  • Physics & Mathematics (AREA)
  • Development Economics (AREA)
  • General Physics & Mathematics (AREA)
  • Marketing (AREA)
  • Health & Medical Sciences (AREA)
  • General Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • General Health & Medical Sciences (AREA)
  • Tourism & Hospitality (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Operations Research (AREA)
  • Computational Linguistics (AREA)
  • Water Supply & Treatment (AREA)
  • Quality & Reliability (AREA)
  • Primary Health Care (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Public Health (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Charge And Discharge Circuits For Batteries Or The Like (AREA)
  • Electric Propulsion And Braking For Vehicles (AREA)

Abstract

The invention provides an electric vehicle charging guidance optimization method based on graph neural network reinforcement learning, which comprises the following steps: step S1: initializing a power-traffic fusion network collaborative optimization model; step S2: updating the charging load of the electric automobile; step S3: generating according to epsilon-Greedy algorithm and graph neural network reinforcement learning algorithma i,t (ii) a Step S4: enforcing charging boot behavior policiesa i,t (ii) a Step S5: calculating a reward function of the graph neural network reinforcement learning algorithm; step S6: partially observing the state of the Markov decision processx i,t Updating; step S7: information of the current step: (x i,t , a i,t ,r i,t ,x i,t ) Is stored in a memory cellDPerforming the following steps; step S8: judging whether the preset time is reachedT end(ii) a If not, executing (2) to (7); if yes, outputting the parameters of the graph neural network reinforcement learning algorithm and corresponding output results. By applying the technical scheme, the total charging cost of the electric automobile can be effectively reduced, and the orderly charging of the electric automobile and the collaborative optimization scheduling of the power system are realized.

Description

Electric vehicle charging guide optimization method based on graph neural network reinforcement learning
Technical Field
The invention relates to the technical field of collaborative optimization of an electric power-traffic fusion network, in particular to an electric vehicle charging guidance optimization method based on graph neural network reinforcement learning.
Background
With the large-scale operation of electric automobiles, a power system and a traffic system have a lot of interactive fusion to form a power-traffic fusion network. The fusion network relates to a plurality of main bodies such as electric automobiles, electric power systems, traffic systems and the like, and comprises a plurality of random uncertain factors. The interaction of a plurality of subjects, the influence of a plurality of random factors and the coupling relationship of the plurality of random factors make the understanding of the interaction influence mechanism of the electric power and traffic system and the solving of the cooperative optimization of the electric power-traffic fusion network more difficult. For example, traveling, psychological behaviors and driving behaviors of electric vehicle users have certain randomness, which affects flow distribution of a traffic system, so that the traffic flow also has certain uncertainty, further affects the time of the electric vehicle reaching a charging station, and causes charging time, queuing time and charging duration of the electric vehicle to have strong uncertainty. Unlike the conventional electric load, the electric vehicle is a movable load, and its randomness is stronger and more difficult to predict than the conventional electric load.
The current research on the electric power-traffic fusion network can be divided into three research directions: 1) from the perspective of an electric power system, the electric vehicle is guided to be charged at the lowest cost by calculating the marginal cost electricity price of the node or optimizing the service pricing of the charging station; 2) the charging cost is minimized by considering the charging path optimization from the perspective of a traffic system; 3) the benefits of the electric automobile, the electric power and the traffic system are comprehensively considered, and the comprehensive benefit maximization is realized by optimizing the charging strategy of the electric automobile and the scheduling decision of the electric power system. However, most of the existing research belongs to a static optimization problem, and the coupling relation of main bodies such as an electric vehicle, a charging station and an electric power system on a continuous time scale is not considered; meanwhile, most of the existing researches do not consider the influence of various uncertain factors and relevant coupling on the collaborative optimization of the power-traffic fusion network. More importantly, the influence of interaction between electric vehicles on the collaborative optimization of the electric power-traffic fusion network is not considered in the existing research.
Disclosure of Invention
In view of this, the present invention provides an electric vehicle charging guidance optimization method based on graph neural network reinforcement learning, which can effectively reduce the total charging cost of an electric vehicle and realize ordered charging of the electric vehicle and collaborative optimization scheduling of an electric power system under the condition of considering multiple uncertainty factors of an electric power-traffic fusion network.
In order to achieve the purpose, the invention adopts the following technical scheme: the electric vehicle charging guidance optimization method based on graph neural network reinforcement learning comprises the following steps:
step S1: initializing a power-traffic fusion network collaborative optimization model;
step S2: updating the charging load of the electric automobile, and carrying out optimization calculation on the marginal cost electricity price of the node where the electric automobile charging station is located based on second-order cone relaxation optimization and dual theory;
step S3: generating an electric vehicle charging guide behavior strategy a according to epsilon-Greedy algorithm and graph neural network reinforcement learning algorithmi,t；
Step S4: enforcing a charging bootstrapping behavior policy ai,tJudging and updating the state of the electric automobile;
step S5: calculating a reward function of a neural network reinforcement learning algorithm according to the electric power-traffic fusion environment;
step S6: partially observing state x of Markov decision processi,tUpdating;
step S7: information (x) of the current stepi,t,ai,t,ri,t,xi,t') storing in a memory unit D, and updating the graph neural network reinforcement learning algorithm weight based on a random gradient descent method; wherein x isi,tShowing the current state of the graph neural network reinforcement learning; a isi,tRepresenting an electric vehicle behavior strategy;ri,ta reward function value representing graph neural network reinforcement learning; x is a radical of a fluorine atomi,t' represents the state of the next step of the graph neural network reinforcement learning;
step S8: judging whether the preset time T is reachedend(ii) a If not, executing (2) - (7); if yes, outputting parameters of the graph neural network reinforcement learning algorithm and corresponding output results.
In a preferred embodiment, initializing the collaborative optimization model of the power-traffic fusion network comprises the following steps:
step 21: determining topological structures and parameters of a power network and a traffic network, wherein the topological structures and the parameters comprise power system nodes, lines, initial voltage and optimized upper and lower limit values, and the traffic network comprises traffic nodes, road parameters, capacity and maximum running speed;
step 22: neural network parameter initialization, including neural network weight initialization and hyper-parameter setting, such as learning rate alpha, discount factor gamma, batch size B and memory unit D capacity;
step 23: regarding each electric automobile in the research area as an agent, regarding the agent as a node N ∈ N, regarding the connection between the electric automobiles as an edge E ∈ E, and forming a graph network structure G ═ N, E, and regarding each electric automobile i in the current state x ∈ N, Ei,tAnd the adjacency matrix a.
In a preferred embodiment, the updating of the charging load of the electric vehicle and the optimization calculation of the marginal cost price of the node where the electric vehicle charging station is located based on the second order cone relaxation optimization and the dual theory comprise:
step 31: updating the charging load of the electric automobile: calculating the charging load of each charging station according to the number of electric vehicles and the charging power in the charging stations, and adding the basic load of the node to the charging load of each station to obtain the final electric load of the node;
step 32: establishing a power distribution network optimal power flow model based on a branch power flow model:
min f(p,q,P,Q,V,I) (1)
Figure BDA0003494780970000031
Figure BDA0003494780970000032
Figure BDA0003494780970000033
Figure BDA0003494780970000034
Figure BDA0003494780970000035
Figure BDA0003494780970000036
Figure BDA0003494780970000037
Figure BDA0003494780970000041
in the formula, ENAnd ELRespectively representing a power distribution network node and a line set; pijAnd QijRepresenting the active power and the reactive power of a branch flowing from the node i to the node j; pjkRepresenting the branch active power flowing from the node j to the node k;
Figure BDA0003494780970000042
and
Figure BDA0003494780970000043
representing active and reactive power of the generator, i.e. active injected into node jPower and reactive power;
Figure BDA0003494780970000044
and
Figure BDA0003494780970000045
representing the active power and the reactive power injected into the node j by the fan; qjsRepresenting the branch reactive power flowing from node j to node s; r isijAnd xijRepresents the branch resistance and reactance from node i to node j; i isijRepresents the branch current from node i to node j; π (j) represents the set of branches connected to node j;
Figure BDA0003494780970000046
and
Figure BDA0003494780970000047
representing the active and reactive loads connected at node j; viRepresents the voltage amplitude of the node i; vjRepresents the voltage magnitude of node j; z is a radical ofijRepresents the impedance of the branch connecting node i and node j, and satisfies zij＝rij+jxij；
Figure BDA0003494780970000048
Representing the maximum value of the branch current connecting the node i and the node j;V jand
Figure BDA0003494780970000049
represents the minimum and maximum voltages of node j;
Figure BDA00034947809700000410
representing the maximum active output of the fan connected to node j;
Figure BDA00034947809700000411
represents a power factor of the fan connected to node j;
load of distribution network node j
Figure BDA00034947809700000412
Including base load
Figure BDA00034947809700000413
And charging load of electric vehicle
Figure BDA00034947809700000414
Namely, it is
Figure BDA00034947809700000415
According to the actual demand of the distribution network, the objective function min f (P, Q, V, I) can be finally defined as:
Figure BDA00034947809700000416
in the formula (I), the compound is shown in the specification,
Figure BDA00034947809700000417
representing the active output of the generator injected into node i; a isiAnd biRespectively representing the secondary coal consumption and the primary coal consumption coefficient of the generator;
Figure BDA00034947809700000418
and
Figure BDA00034947809700000419
respectively purchasing the electricity price and the active power of the electric quantity from the main network;
step 33: converting the optimal power flow model of the nonlinear power distribution network into a second-order cone relaxation planning model:
the BFM-OPF is a nonlinear programming model, so that the branch current amplitude value
Figure BDA00034947809700000420
And branch voltage amplitude
Figure BDA00034947809700000421
And performing a second order taper relaxation (SOCR) conversion on the equationAlternatively, the following model can be obtained:
Figure BDA0003494780970000051
Figure BDA0003494780970000052
Figure BDA0003494780970000053
Figure BDA0003494780970000054
Figure BDA0003494780970000055
Figure BDA0003494780970000056
Figure BDA0003494780970000057
in the formula | · | non-conducting phosphor2Representing a second order cone operation; the formula constitutes the basic form of the optimal power flow of the power distribution network after relaxation;
step 34: solving the original problem and the dual variable of the model by adopting a Gurobi solver to obtain the marginal cost electrovalence lambda of the node where the charging station is locatedk。
In a preferred embodiment, the epsilon-Greedy algorithm comprises the steps of:
step 41: generating a random number u, and judging the size of the random number u and a decay factor xi of the epsilon-Greedy algorithm;
step 42: if u<Xi, then adopting random mode to treat each vehicle in current stateThe electric automobile generates a behavior ai,tThis behavior is expressed in the patent as an electric vehicle charging path strategy;
ai,t＝randint(Naction) (19)
in the formula, NactionRepresenting the number of electric vehicle behavior decisions;
step 43: if u is larger than or equal to xi, each electric automobile i is in the current state x according to experience of the graph neural network reinforcement learning algorithmi,tAnd generating a behavior a under the adjacency matrix Ai,tI.e. by
Figure BDA0003494780970000058
In the formula, thetatParameters representing a graph neural network reinforcement learning algorithm; argmax () represents a parameter operation corresponding to the maximum value; x is the number ofi,tThe state of the ith electric vehicle at the time t is shown, which is mainly composed of the state x of the ith electric vehicle at the time ti,tFrom the electric vehicle state EVi,tAnd adjacent traffic road information Roi,tState Ne of electric vehicle in close proximityi,tAnd each charging station information CStIs composed of, i.e.
xi,t＝[EVi,t,Roi,t,Nei,t,CSt] (21)
Figure BDA0003494780970000061
Figure BDA0003494780970000062
Figure BDA0003494780970000063
Figure BDA0003494780970000064
In the formula, i-th electric vehicle state EVi,tIncluding the next node when the electric vehicle goes to the charging station
Figure BDA0003494780970000065
Road numbering
Figure BDA0003494780970000066
Electric automobile driving speed vi,tAnd remaining capacity SOCi,t(ii) a Neighbor traffic road information state Roi,tComprises the next node of the electric automobile i
Figure BDA0003494780970000067
Starting node of the next road connected
Figure BDA0003494780970000068
End node
Figure BDA0003494780970000069
Road length
Figure BDA00034947809700000610
And number of electric vehicles on road
Figure BDA00034947809700000611
Nearest neighbor electric vehicle state Nei,tIncluding the status of each neighboring electric vehicle k, e.g. the next node of the kth electric vehicle adjacent to the ith electric vehicle
Figure BDA00034947809700000612
The road number of the road
Figure BDA00034947809700000613
Electric automobile driving speed vi,k,tAnd remaining capacity SOCi,k,t(ii) a Charging station information CStCharging tariff p comprising charging stationsc,tAnd number of electric vehicles
Figure BDA00034947809700000614
The graph neural network reinforcement learning algorithm comprises a neural network structure comprising an input layer, a full connection layer and an input state xi,tPerforming feature extraction xi,t', then the characteristic x to be proposedi,tInputting the two layers of the adjacent matrixes A into a two-layer graph neural network together, extracting features, and finally connecting a full-connection layer to carry out a strategy a of charging paths of the electric automobilei,tOutputting; wherein, the graph neural network adopts a graph attention network.
In a preferred embodiment, the reward function r of the graph neural network reinforcement learning algorithmi,tAs shown in the formula:
Figure BDA0003494780970000071
in the formula, a nodecurAnd a nodetarIndicating the current node of the electric automobile and any charging station node to which the electric automobile is going, and step indicating the number of steps the electric automobile has run; penalty represents a large penalty factor; w is aiIs shown asiThe cost per unit time of the electric vehicle;
Figure BDA0003494780970000072
and
Figure BDA0003494780970000073
respectively representing the driving time, the charging waiting time and the charging required time when the ith electric vehicle goes to the kth charging station at the time t; lambda [ alpha ]k,tRepresenting the marginal cost price of the node where the charging station k is located at the time t; SOCi,k,tRepresents the remaining capacity SOC when the ith electric vehicle reaches the charging station k at the time ti,k,t；
Figure BDA0003494780970000074
The battery rated capacity of the ith electric automobile is represented;
the reward function can be seen from the formulari,tIs a piecewise function; if the ith electric vehicle does not reach the charging station nodecur≠nodetarAnd the step number of the current electric vehicle going to the charging station is step < N within the given maximum charging step numberstepAt this time, its reward function ri,t0; if the number of steps of the ith electric vehicle going to the charging station is more than or equal to the given maximum charging step number step which is more than or equal to NstepWhen the charging behavior is failed to be explored, a larger negative reward r is given to the charging behaviori,t-penalty; if the ith electric vehicle arrives at the charging station nodecur＝nodetarAnd the step number of the current electric vehicle going to the charging station is step < N within the given maximum charging step numberstepAt the moment, the reward function is based on the running time of the electric automobile
Figure BDA0003494780970000075
And charging time
Figure BDA0003494780970000076
And calculating the electricity charge during charging;
the passing time t of the ith electric automobile on the road section aa,tCalculated according to the U.S. Federal road administration function (BPR), i.e. the buseau of public roads
Figure BDA0003494780970000077
In the formula, na,tRepresenting the number of electric vehicles on the road section a at the moment t; c. CaAnd
Figure BDA0003494780970000078
respectively representing the upper limit of the capacity of the road section a and the free passing time of the electric automobile at the time t; therefore, the time required for the ith electric vehicle to go to the charging station k can be obtained
Figure BDA0003494780970000081
Namely that
Figure BDA0003494780970000082
Further, the charging wait time of the ith electric vehicle
Figure BDA0003494780970000083
Can be obtained by the formula;
Figure BDA0003494780970000084
in the formula, SOCtRepresenting the residual electric quantity of the electric automobile;
Figure BDA0003494780970000085
the rated capacity of the battery of the electric automobile is represented; eta represents a charging power factor, PchargingIndicating the rated power for charging the electric vehicle.
In a preferred embodiment, the updating weights of the graph neural network reinforcement learning algorithm based on the stochastic gradient descent method comprises:
step 61: randomly extracting a certain number of samples from the memory unit D;
step 62: constructing a loss function as shown in the formula, and updating the weight of the graph neural network reinforcement learning algorithm according to a random gradient descent method under the extracted Sample as shown in the formula;
Figure BDA0003494780970000086
wherein x, a, x 'and a' are the current state, action and the state and action at the next moment, respectively; r represents the immediate reward of graph neural network reinforcement learning; thetatA graph neural network reinforcement learning algorithm parameter representing the current time t; gamma is greater than or equal to 0 and less than or equal to 1, representing a discount factor that reflects the impact of future Q values on the current action;
Figure BDA0003494780970000087
representing the neural network reinforcement learning calculation in the target graphMethod parameter theta'tState-action value of;
Figure BDA0003494780970000088
in the formula, thetatA graph neural network reinforcement learning algorithm parameter representing the current time t;
Figure BDA0003494780970000089
is expressed in the pair thetatCarrying out derivation operation; α represents a learning rate;
and step 63: the neural network reinforcement learning parameter theta is obtained according to the current graph after a certain number of stepstReinforcing learning parameter theta 'for neural network of target map'tAnd (6) updating.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides an electric vehicle charging guidance optimization method based on graph neural network reinforcement learning, which converts the mutual influence relationship among electric vehicles into a dynamic network graph structure based on a graph theory, and provides an attention mechanism-based graph neural network reinforcement learning method for processing irregular non-European structural data so as to research the communication and cooperation among multiple intelligent agents and discuss the mutual influence among the electric vehicles. On the basis of considering the output of renewable energy sources of an active power distribution network, the optimal power flow of the power distribution network is solved through a second-order cone optimization and dual optimization theory, and the marginal cost price of the power distribution network node is obtained, so that the power-traffic fusion network collaborative optimization is researched. The electric vehicle charging guiding optimization method based on graph neural network reinforcement learning can effectively reduce the total charging cost of the electric vehicle and realize ordered charging of the electric vehicle and cooperative optimization scheduling of a power system under the condition that multiple uncertainty factors of a power-traffic fusion network are considered.
Drawings
Fig. 1 is a flowchart of an electric vehicle charging guidance optimization method based on graph neural network reinforcement learning according to a preferred embodiment of the present invention.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application; as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
As shown in fig. 1, the invention relates to an electric vehicle charging guidance optimization method based on graph neural network reinforcement learning, which includes the following steps:
s11: initializing a power-traffic fusion network collaborative optimization model;
s12: updating the charging load of the electric automobile, and carrying out optimization calculation on the marginal cost electricity price of the node where the electric automobile charging station is located based on second-order cone relaxation optimization and dual theory;
s13: generating an electric vehicle charging guide behavior strategy a according to epsilon-Greedy algorithm and graph neural network reinforcement learning algorithmi,t；
S14: enforcing a charging bootstrapping behavior policy ai,tJudging and updating the state of the electric automobile;
s15: calculating a reward function of a neural network reinforcement learning algorithm according to the electric power-traffic fusion environment;
s16: partially observing state x of Markov decision processi,tUpdating;
s17: information (x) of the current stepi,t,ai,t,ri,t,xi,t') stored in memory cells D and based on randomUpdating the weight of the graph neural network reinforcement learning algorithm by a gradient descent method;
s18: judging whether the preset time T is reachedend. If not, executing (2) - (7); if yes, outputting the parameters of the graph neural network reinforcement learning algorithm and corresponding output results.
Specifically, the method comprises the following steps:
firstly, initializing a power-traffic fusion network collaborative optimization model. The method mainly comprises the steps of determining topological structures and parameters of the power network and the traffic network, wherein the topological structures and the parameters comprise power system nodes, lines, initial voltage and optimized upper and lower limit values, and the traffic network comprises traffic nodes, road parameters, capacity, maximum driving speed and the like.
Neural network parameter initialization, including neural network weight initialization and hyper-parameter setting, such as learning rate alpha, discount factor gamma, batch size B and memory unit capacity D;
regarding each electric automobile in the research area as an agent, regarding the agent as a node N ∈ N, regarding the connection between the electric automobiles as an edge E ∈ E, and forming a graph network structure G ═ N, E, and regarding each electric automobile i in the current state x ∈ N, Ei,tAnd the adjacency matrix a.
And secondly, updating the charging load of the electric automobile, and performing optimization calculation on the marginal cost electricity price of the node where the electric automobile charging station is located based on second-order cone relaxation optimization and dual theory. The method mainly comprises the following steps:
step 21: updating the charging load of the electric automobile: calculating the charging load of each charging station according to the number of electric vehicles and the charging power in the charging stations, and adding the basic load of the node to the charging load of each station to obtain the final electric load of the node;
step 22: establishing a power distribution network optimal power flow model based on a branch power flow model:
min f(p,q,P,Q,V,I) (1)
Figure BDA0003494780970000111
Figure BDA0003494780970000112
Figure BDA0003494780970000113
Figure BDA0003494780970000114
Figure BDA0003494780970000115
Figure BDA0003494780970000116
Figure BDA0003494780970000117
Figure BDA0003494780970000118
in the formula, ENAnd ELRespectively representing a power distribution network node and a line set; pijAnd QijRepresenting the active power and the reactive power of a branch flowing from the node i to the node j;
Figure BDA0003494780970000119
and
Figure BDA00034947809700001110
representing the active and reactive outputs of the generator, i.e. the active and reactive power injected into node j;
Figure BDA00034947809700001111
and
Figure BDA00034947809700001112
representing active power and reactive power injected into the node j by the fan; r isijAnd xijRepresents the branch resistance and reactance from node i to node j; I.C. AijRepresents the branch current from node i to node j; π (j) represents the set of branches connected to node j;
Figure BDA0003494780970000121
and
Figure BDA0003494780970000122
representing the active and reactive loads connected at node j; viRepresents the voltage amplitude of the node i; z is a radical ofijRepresents the impedance of the branch connecting node i and node j, and satisfies zij＝rij+jxij；
Figure BDA0003494780970000123
Representing the maximum value of the branch current connecting the node i and the node j;V jand
Figure BDA0003494780970000124
represents the minimum and maximum voltages of node j;
Figure BDA0003494780970000125
representing the maximum active output of the fan connected to node j;
Figure BDA0003494780970000126
representing the power factor of the fan connected to node j.
Load of distribution network node j
Figure BDA0003494780970000127
Including base load
Figure BDA0003494780970000128
And charging load of electric vehicle
Figure BDA0003494780970000129
Namely, it is
Figure BDA00034947809700001210
According to the actual demand of the distribution network, the objective function min f (P, Q, V, I) can be finally defined as:
Figure BDA00034947809700001211
in the formula, aiAnd biRespectively representing the secondary coal consumption and the primary coal consumption coefficient of the generator;
Figure BDA00034947809700001212
and
Figure BDA00034947809700001213
the electricity price and the active power of the electric quantity are respectively purchased from the main network.
Step 23, converting the optimal power flow model of the nonlinear power distribution network into a second-order cone relaxation planning model:
since BFM-OPF is a nonlinear programming model, let
Figure BDA00034947809700001214
And
Figure BDA00034947809700001215
and performing second-order cone relaxation (SOCR) conversion on the formula to obtain the following model:
Figure BDA00034947809700001216
Figure BDA00034947809700001217
Figure BDA00034947809700001218
Figure BDA00034947809700001219
Figure BDA0003494780970000131
Figure BDA0003494780970000132
Figure BDA0003494780970000133
in the formula | · | non-conducting phosphor2Representing a second order cone operation; the above formula constitutes the basic form of the optimal power flow of the power distribution network after relaxation.
Step 24, solving the original problem and the dual variable of the model by adopting a Gurobi solver, and obtaining the marginal cost electrovalence lambda of the node where the charging station is locatedk。
Thirdly, generating an electric vehicle charging guide behavior strategy a according to the epsilon-Greedy algorithm and the graph neural network reinforcement learning algorithmi,t. The method mainly comprises the following steps:
step 31: and generating a random number u, and judging the size of the random number u and a decay factor xi of the epsilon-Greedy algorithm.
Step 32: if u<Xi, generating a behavior a for each electric automobile in the current state in a random modei,tThis behavior is expressed in the patent as an electric vehicle charging path strategy;
ai,t＝randint(Naction) (19)
in the formula, NactionRepresenting the number of electric vehicle behavior decisions.
Step 33: if u is larger than or equal to xi, each electric automobile i is subjected to experience of the graph neural network reinforcement learning algorithmAt the current state xi,tAnd generating a behavior a under the adjacency matrix Ai,tI.e. by
Figure BDA0003494780970000134
In the formula, thetatParameters representing a graph neural network reinforcement learning algorithm; argmax () represents a parameter operation corresponding to the maximum value; x is the number ofi,tThe state of the ith electric vehicle at the time t is shown, which is mainly composed of the state x of the ith electric vehicle at the time ti,tFrom the electric vehicle state EVi,tAnd adjacent traffic road information Roi,tState Ne of neighboring electric vehiclei,tAnd each charging station information CStIs composed of, i.e.
xi,t＝[EVi,t,Roi,t,Nei,t,CSt] (21)
Figure BDA0003494780970000135
Figure BDA0003494780970000141
Figure BDA0003494780970000142
Figure BDA0003494780970000143
In the formula, i-th electric vehicle state EVi,tIncluding the next node when the electric vehicle goes to the charging station
Figure BDA0003494780970000144
Road numbering
Figure BDA0003494780970000145
Electric steamVehicle running speed vi,tAnd remaining capacity SOCi,t(ii) a Neighbor traffic road information state Roi,tComprises the next node of the electric automobile i
Figure BDA0003494780970000146
Starting node of the next road connected
Figure BDA0003494780970000147
End node
Figure BDA0003494780970000148
Road length
Figure BDA0003494780970000149
And number of electric vehicles on road
Figure BDA00034947809700001410
Nearest neighbor electric vehicle state Nei,tIncluding the status of each neighboring electric vehicle k, e.g. the next node of the kth electric vehicle adjacent to the ith electric vehicle
Figure BDA00034947809700001411
The road number of the road
Figure BDA00034947809700001412
Electric automobile driving speed vi,k,tAnd remaining capacity SOCi,k,t(ii) a Charging station information CStCharging tariff p comprising charging stationsc,tAnd number of electric vehicles
Figure BDA00034947809700001413
The graph neural network reinforcement learning algorithm comprises a neural network structure comprising an input layer, a full connection layer and an input state xi,tPerforming feature extraction xi,t', then the characteristic x to be proposedi,tInputting the two layers of the adjacent matrixes A into a two-layer graph neural network together, extracting features, and finally connecting a layer of full-connection layer to control the charging path of the electric automobileA little bit lessi,tAnd outputting the data. The graph neural network described in this patent employs a graph attention network.
Fourthly, executing a charging guide action strategy ai,tAnd the state of the electric automobile is judged and updated. The state of the electric vehicle is classified into three types: decision state, running state and charging state. If the electric automobile arrives at the intersection nodecur＝nodenextAnd the intersection is not a charging station nodecur≠nodetarWhen the electric vehicle is in a decision-making state, the electric vehicle executes a charging guide action strategy ai,tUpdating road states such as the number of electric vehicles and ideal driving speed, and updating information such as the positions of the roads where the electric vehicles are located, driving speed, distance and the like; if the electric automobile does not reach the intersection nodecur≠nodenextWhen the electric vehicle is in the running state, namely the electric vehicle follows the charging guide strategy a of the previous stepi,t-1Continuing to drive forwards along the current road, and updating the position information, the speed information and the SOC state of the electric vehicle at the moment; if node on the charging station node at the node position of the electric automobilecur＝nodetarAnd at the moment, the electric automobile is in a charging state, if the number of the current electric automobiles is larger than the number of the charging piles in the charging station, the electric automobiles need to wait in a queue for charging, if available charging piles in the charging station are used, the electric automobiles immediately charge, and the charging waiting time, the charging time and the SOC state of the electric automobiles are updated.
And fifthly, calculating a reward function of the neural network reinforcement learning algorithm according to the power-traffic fusion environment. In particular, the reward function ri,tIs a piecewise function: if the ith electric vehicle does not reach the charging station nodecur≠nodetarAnd the step number of the current electric vehicle going to the charging station is step < N within the given maximum charging step numberstepAt this time, its reward function ri,t0; if the number of steps of the ith electric vehicle going to the charging station is more than or equal to the given maximum charging step number step which is more than or equal to NstepWhen the charging action fails to be explored, a larger negative prize is given to the charging actionExciteri,t-penalty; if the ith electric vehicle arrives at the charging station nodecur＝nodetarAnd the step number of the current electric vehicle going to the charging station is step < N within the given maximum charging step numberstepWhen the reward function is according to the driving time of the electric automobile
Figure BDA0003494780970000151
Charging waiting time
Figure BDA0003494780970000152
Charging time
Figure BDA0003494780970000153
And calculating the electricity charge during charging, wherein the specific calculation expression is shown.
Figure BDA0003494780970000154
Travel time
Figure BDA0003494780970000155
Charging waiting time
Figure BDA0003494780970000156
Charging time
Figure BDA0003494780970000157
The calculation expression is shown as.
The passing time of the ith electric vehicle on the road section a is calculated according to the U.S. Federal road administration function (BPR), namely
Figure BDA0003494780970000158
In the formula, na,tRepresenting the number of electric vehicles on the road section a at the moment t; c. CaAnd
Figure BDA0003494780970000159
and respectively representing the upper capacity limit of the road section a and the free passing time of the electric automobile at the time t. Therefore, the time required for the ith electric vehicle to go to the charging station k can be obtained
Figure BDA00034947809700001510
Namely, it is
Figure BDA00034947809700001511
Further, charging wait time of ith electric vehicle
Figure BDA0003494780970000161
Can be obtained by the following formula.
Figure BDA0003494780970000162
In the formula, SOCtRepresenting the residual capacity of the electric automobile;
Figure BDA0003494780970000163
the rated capacity of the battery of the electric automobile is represented; eta represents a charging power factor, PchargingIndicating the rated power for charging the electric vehicle.
Sixthly, partially observing state x of Markov decision processi,tUpdating, including updating electric vehicle state EVi,tAnd adjacent traffic road information Roi,tState Ne of electric vehicle in close proximityi,tAnd each charging station information CSt。
Seventhly, information (x) of the current step is obtainedi,t,ai,t,ri,t,xi,t') is stored in the memory unit D, and the weights of the graph neural network reinforcement learning algorithm are updated based on the stochastic gradient descent method. The method mainly comprises the following steps:
step 71: randomly extracting a certain number of samples from the memory unit D;
step 72: constructing a loss function as shown in the formula, and updating the weight of the graph neural network reinforcement learning algorithm according to a random gradient descent method under the extracted Sample as shown in the formula;
Figure BDA0003494780970000164
wherein x, a, x 'and a' are the current state, action and the state and action at the next moment, respectively; thetatA graph neural network reinforcement learning algorithm parameter representing the current time t; gamma is greater than or equal to 0 and less than or equal to 1, representing a discount factor that reflects the impact of future Q values on the current action;
Figure BDA0003494780970000165
representing parameter theta 'of neural network reinforcement learning algorithm in target graph'tState of state-action value.
Figure BDA0003494780970000166
In the formula, thetatA graph neural network reinforcement learning algorithm parameter representing the current time t;
Figure BDA0003494780970000167
is expressed in the pair thetatCarrying out derivation operation; α represents a learning rate.
Step 73: the neural network reinforcement learning parameter theta is obtained according to the current graph after a certain number of stepstReinforcing learning parameter theta 'for neural network of target map'tAnd (6) updating.
Eighthly, judging whether the preset time T is reachedend. If not, executing (2) - (7); if yes, outputting the parameters of the graph neural network reinforcement learning algorithm and corresponding output results.
The invention discloses an electric vehicle charging guiding optimization method based on graph neural network reinforcement learning, which converts the mutual influence relation among electric vehicles into a dynamic network graph structure based on a graph theory, provides an attention mechanism-based graph neural network reinforcement learning method for processing irregular non-European structural data, and researches the communication and cooperation among multiple intelligent agents to discuss the mutual influence among the electric vehicles. On the basis of considering the active power distribution network of renewable energy output, the optimal power flow of the power distribution network is solved through a second-order cone optimization theory and a dual optimization theory, and the marginal cost electricity price of the power distribution network node is obtained, so that the power-traffic fusion network collaborative optimization is researched. The electric vehicle charging guiding optimization method based on graph neural network reinforcement learning can effectively reduce the total charging cost of the electric vehicle and realize ordered charging of the electric vehicle and cooperative optimization scheduling of a power system under the condition that multiple uncertainty factors of a power-traffic fusion network are considered.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (6)

1. The electric vehicle charging guidance optimization method based on graph neural network reinforcement learning is characterized by comprising the following steps:
step S1: initializing a power-traffic fusion network collaborative optimization model;
step S2: updating the charging load of the electric automobile, and carrying out optimization calculation on the marginal cost electricity price of the node where the electric automobile charging station is located based on second-order cone relaxation optimization and dual theory;
step S3: generating an electric vehicle charging guide behavior strategy a according to epsilon-Greedy algorithm and graph neural network reinforcement learning algorithmi,t；
Step S4: enforcing a charging bootstrapping behavior policy ai,tJudging and updating the state of the electric automobile;
step S5: reward function r of neural network reinforcement learning algorithm according to electric power-traffic fusion environment calculation diagrami,t；
Step S6: partially observing state x of Markov decision processi,tUpdating;
step S7: information (x) of the current stepi,t,ai,t,ri,t,xi,t') storing in a memory unit D, and updating the graph neural network reinforcement learning algorithm weight based on a random gradient descent method; wherein x isi,tShowing the current state of the graph neural network reinforcement learning; a isi,tRepresenting an electric vehicle behavior strategy; r isi,tThe reward function value represents the graph neural network reinforcement learning; x is the number ofi,t' represents the state of the next step of the graph neural network reinforcement learning;
step S8: judging whether the preset time T is reachedend(ii) a If not, executing (2) - (7); if yes, outputting the parameters of the graph neural network reinforcement learning algorithm and corresponding output results.
2. The electric vehicle charging guidance optimization method based on graph neural network reinforcement learning according to claim 1, wherein initializing a power-traffic fusion network collaborative optimization model comprises the following steps:
step 21: determining topological structures and parameters of a power network and a traffic network, wherein the topological structures and the parameters comprise power system nodes, lines, initial voltage and optimized upper and lower limit values, and the traffic network comprises traffic nodes, road parameters, capacity and maximum running speed;
step 22: neural network parameter initialization, including neural network weight initialization and hyper-parameter setting, such as learning rate alpha, discount factor gamma, batch size B and memory unit D capacity;
step 23: regarding each electric automobile in the research area as an agent, regarding the agent as a node N belongs to N, regarding the connection between the electric automobiles as an edge E belongs to E, forming a graph network structure G-N, E, and regarding each electric automobile i in the current state xi,tAnd the adjacency matrix a.
3. The electric vehicle charging guidance optimization method based on graph neural network reinforcement learning of claim 1, wherein the steps of updating the electric vehicle charging load and performing optimization calculation on the marginal cost electricity price of the node where the electric vehicle charging station is located based on the second-order cone relaxation optimization and the dual theory comprise:
step 31: updating the charging load of the electric automobile: calculating the charging load of each charging station according to the number of electric vehicles and the charging power in the charging stations, and adding the basic load of the node to the charging load of each station to obtain the final electric load of the node;
step 32: establishing a power distribution network optimal power flow model based on a branch power flow model:
minf(p,q,P,Q,V,I) (1)
s.t.
Figure FDA0003494780960000021
Figure FDA0003494780960000022
Figure FDA0003494780960000023
Figure FDA0003494780960000024
Figure FDA0003494780960000025
Figure FDA0003494780960000026
Figure FDA0003494780960000027
Figure FDA0003494780960000028
in the formula, ENAnd ELRespectively representing a power distribution network node and a line set; pijAnd QijRepresenting the active power and the reactive power of a branch flowing from the node i to the node j; pjkRepresenting the branch active power flowing from the node j to the node k;
Figure FDA0003494780960000031
and
Figure FDA0003494780960000032
representing the active and reactive outputs of the generator, i.e. the active and reactive power injected into node j;
Figure FDA0003494780960000033
and
Figure FDA0003494780960000034
representing the active power and the reactive power injected into the node j by the fan; qjsRepresenting the branch reactive power flowing from node j to node s; r isijAnd xijRepresents the branch resistance and reactance from node i to node j; i isijRepresents the branch current from node i to node j; π (j) represents the set of branches connected to node j;
Figure FDA0003494780960000035
and
Figure FDA0003494780960000036
representing the active and reactive loads connected at node j; viRepresents the voltage amplitude of the node i; vjRepresents the voltage magnitude of node j; z is a radical ofijShow the connecting jointThe branch impedance of the point i and the node j satisfies zij＝rij+jxij；
Figure FDA0003494780960000037
Representing the maximum value of the branch current connecting the node i and the node j;V jand
Figure FDA0003494780960000038
represents the minimum and maximum voltages of node j;
Figure FDA0003494780960000039
representing the maximum active output of the fan connected to node j;
Figure FDA00034947809600000310
represents a power factor of the fan connected to node j;
load of distribution network node j
Figure FDA00034947809600000311
Including base load
Figure FDA00034947809600000312
And charging load of electric vehicle
Figure FDA00034947809600000313
Namely, it is
Figure FDA00034947809600000314
According to the actual demand of the distribution network, the objective function minf (P, Q, V, I) can be finally defined as:
Figure FDA00034947809600000315
in the formula, aiAnd biRespectively representThe secondary coal consumption and the primary coal consumption coefficient of the generator;
Figure FDA00034947809600000316
representing the active output of the generator injected into node i;
Figure FDA00034947809600000317
and
Figure FDA00034947809600000318
respectively purchasing the electricity price and the active power of the electric quantity from the main network;
step 33: converting the optimal power flow model of the nonlinear power distribution network into a second-order cone relaxation planning model:
the BFM-OPF is a nonlinear programming model, so that the branch current amplitude value
Figure FDA00034947809600000319
And branch voltage amplitude
Figure FDA00034947809600000320
And performing second-order cone relaxation (SOCR) conversion on the formula to obtain the following model:
Figure FDA00034947809600000321
s.t.
Figure FDA00034947809600000322
Figure FDA0003494780960000041
Figure FDA0003494780960000042
Figure FDA0003494780960000043
Figure FDA0003494780960000044
Figure FDA0003494780960000045
in the formula | · | non-conducting phosphor2Representing a second order cone operation; the formula constitutes the basic form of the optimal power flow of the power distribution network after relaxation;
step 34: solving the original problem and the dual variable of the model by adopting a Gurobi solver to obtain the marginal cost electrovalence lambda of the node where the charging station is locatedk。
4. The electric vehicle charging guidance optimization method based on graph neural network reinforcement learning according to claim 3, characterized in that the epsilon-Greedy algorithm comprises the following steps:
step 41: generating a random number u, and judging the size of the random number u and a decay factor xi of the epsilon-Greedy algorithm;
step 42: if u<Xi, generating a behavior a for each electric automobile in the current state in a random modei,tThis behavior is expressed in the patent as an electric vehicle charging path strategy;
ai,t＝randint(Naction) (19)
in the formula, NactionRepresenting the number of electric vehicle behavior decisions;
step 43: if u is larger than or equal to xi, each electric automobile i is in the current state x according to experience of the graph neural network reinforcement learning algorithmi,tAnd generating a behavior a under the adjacency matrix Ai,tI.e. by
Figure FDA0003494780960000046
In the formula, thetatParameters representing a graph neural network reinforcement learning algorithm; argmax () represents a parameter operation corresponding to the maximum value; x is the number ofi,tThe state of the ith electric vehicle at the time t is shown, which is mainly composed of the state x of the ith electric vehicle at the time ti,tFrom the electric vehicle state EVi,tAnd adjacent traffic road information Roi,tState Ne of electric vehicle in close proximityi,tAnd each charging station information CStIs composed of, i.e.
xi,t＝[EVi,t,Roi,t,Nei,t,CSt] (21)
Figure FDA0003494780960000051
Figure FDA0003494780960000052
Figure FDA0003494780960000053
Figure FDA0003494780960000054
In the formula, i-th electric vehicle state EVi,tIncluding the next node when the electric vehicle goes to the charging station
Figure FDA0003494780960000055
Road numbering
Figure FDA0003494780960000056
Electric automobile driving speed vi,tAnd remainderElectric quantity SOCi,t(ii) a Neighbor traffic road information state Roi,tComprises the next node of the electric automobile i
Figure FDA0003494780960000057
Starting node of the next road connected
Figure FDA0003494780960000058
End node
Figure FDA0003494780960000059
Road length
Figure FDA00034947809600000510
And number of electric vehicles on road
Figure FDA00034947809600000511
Nearest neighbor electric vehicle state Nei,tIncluding the status of each neighboring electric vehicle k, e.g. the next node of the kth electric vehicle adjacent to the ith electric vehicle
Figure FDA00034947809600000512
The road number of the road
Figure FDA00034947809600000513
Electric automobile driving speed vi,k,tAnd remaining capacity SOCi,k,t(ii) a Charging station information CStCharging tariff p comprising charging stationsc，tAnd number of electric vehicles
Figure FDA00034947809600000514
The graph neural network reinforcement learning algorithm comprises a neural network structure comprising an input layer, a full connection layer and an input state xi,tPerforming feature extraction xi,t', then the characteristic x to be proposedi,t' inputting the two-layer graph neural network together with the adjacency matrix A and then carrying out feature extractionTaking and finally connecting a full connecting layer to charge the electric automobilei,tOutputting is carried out; wherein, the graph neural network adopts a graph attention network.
5. The electric vehicle charging guidance optimization method based on graph neural network reinforcement learning of claim 2, characterized in that the reward function r of the graph neural network reinforcement learning algorithmi,tAs shown in the formula:
Figure FDA00034947809600000515
Figure FDA0003494780960000061
in the formula, a nodecurAnd a nodetarIndicating the current node of the electric automobile and any charging station node to which the electric automobile is going, and step indicating the number of steps the electric automobile has run; penalty represents a large penalty factor; w is aiIs shown asiCost per unit time of the electric vehicle;
Figure FDA0003494780960000062
and
Figure FDA0003494780960000063
respectively representing the driving time, the charging waiting time and the charging required time when the ith electric vehicle goes to the kth charging station at the time t; lambdak,tRepresenting the marginal cost price of the node where the charging station k is located at the time t; SOCi,k,tRepresents the remaining capacity SOC when the ith electric vehicle reaches the charging station k at the time ti,k,t；
Figure FDA0003494780960000064
The battery rated capacity of the ith electric automobile is represented;
slave typeThe reward function r can be seeni,tIs a piecewise function; if the ith electric vehicle does not reach the charging station nodecur≠nodetarAnd the step number of the current electric vehicle going to the charging station is step < N within the given maximum charging step numberstepAt this time, its reward function ri,t0; if the step number of the ith electric vehicle going to the charging station is more than or equal to the given maximum charging step number step which is more than or equal to NstepWhen the charging behavior is failed to be explored, a larger negative reward r is given to the charging behaviori,t-penalty; if the ith electric vehicle arrives at the charging station nodecur＝nodetarAnd the step number of the current electric vehicle going to the charging station is step < N within the given maximum charging step numberstepWhen the reward function is according to the driving time of the electric automobile
Figure FDA0003494780960000065
And charging time
Figure FDA0003494780960000066
And calculating the electricity charge during charging;
the passing time t of the ith electric automobile on the road section aa,tCalculated according to the U.S. Federal road administration function (BPR), i.e. the buseau of public roads
Figure FDA0003494780960000067
In the formula, na,tRepresenting the number of electric vehicles on the road section a at the moment t; c. CaAnd
Figure FDA0003494780960000068
respectively representing the upper limit of the capacity of the road section a and the free passing time of the electric automobile at the time t; therefore, the time required for the ith electric vehicle to go to the charging station k can be obtained
Figure FDA0003494780960000069
Namely that
Figure FDA00034947809600000610
Further, charging wait time of ith electric vehicle
Figure FDA00034947809600000611
Can be obtained by the formula;
Figure FDA0003494780960000071
in the formula, SOCtRepresenting the residual electric quantity of the electric automobile;
Figure FDA0003494780960000072
the rated capacity of the battery of the electric automobile is represented; eta represents a charging power factor, PchargingIndicating the rated power for charging the electric vehicle.
6. The electric vehicle charging guidance optimization method based on graph neural network reinforcement learning of claim 1, wherein the updating of the graph neural network reinforcement learning algorithm weight based on the stochastic gradient descent method comprises:
step 61: randomly extracting a certain number of samples from the memory unit D;
step 62: constructing a loss function as shown in the formula, and updating the weight of the graph neural network reinforcement learning algorithm according to a random gradient descent method under the extracted Sample as shown in the formula;
Figure FDA0003494780960000073
wherein x, a, x 'and a' are the current state, action and the state and action at the next moment, respectively; r represents the immediate reward of graph neural network reinforcement learning; thetatIndicates the currentA graph neural network reinforcement learning algorithm parameter at the moment t; gamma is greater than or equal to 0 and less than or equal to 1, representing a discount factor that reflects the impact of future Q values on the current action;
Figure FDA0003494780960000074
representing parameter theta 'of neural network reinforcement learning algorithm in target graph'tState-action value of;
Figure FDA0003494780960000075
in the formula, thetatA graph neural network reinforcement learning algorithm parameter representing the current time t;
Figure FDA0003494780960000076
is expressed in the pair thetatCarrying out derivation operation; α represents a learning rate;
and step 63: the neural network reinforcement learning parameter theta is obtained according to the current graph after a certain number of stepstReinforcing learning parameter theta 'for neural network of target map'tAnd (6) updating.
CN202210109887.2A 2022-01-29 2022-01-29 Electric vehicle charging guide optimization method based on graph neural network reinforcement learning Active CN114444802B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210109887.2A CN114444802B (en) 2022-01-29 2022-01-29 Electric vehicle charging guide optimization method based on graph neural network reinforcement learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210109887.2A CN114444802B (en) 2022-01-29 2022-01-29 Electric vehicle charging guide optimization method based on graph neural network reinforcement learning

Publications (2)

Publication Number Publication Date
CN114444802A true CN114444802A (en) 2022-05-06
CN114444802B CN114444802B (en) 2024-06-04

Family

ID=81372174

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210109887.2A Active CN114444802B (en) 2022-01-29 2022-01-29 Electric vehicle charging guide optimization method based on graph neural network reinforcement learning

Country Status (1)

Country Link
CN (1) CN114444802B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116436019A (en) * 2023-04-12 2023-07-14 国网江苏省电力有限公司电力科学研究院 Multi-resource coordination optimization method, device and storage medium
CN118098000A (en) * 2024-04-24 2024-05-28 哈尔滨华鲤跃腾科技有限公司 Urban comprehensive management method based on artificial intelligence
CN118438918A (en) * 2024-05-09 2024-08-06 烟台开发区德联软件有限责任公司 Mobile energy storage device charging regulation and control method and device
WO2024165229A1 (en) * 2023-02-08 2024-08-15 E.On Se Edge computing with ai at mains supply network interconnection points

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110570050A (en) * 2019-09-25 2019-12-13 国网浙江省电力有限公司经济技术研究院 Road-network-vehicle-related electric vehicle charging guiding method
TWI687785B (en) * 2019-02-25 2020-03-11 華碩電腦股份有限公司 Method of returning to charging station
CN111934335A (en) * 2020-08-18 2020-11-13 华北电力大学 Cluster electric vehicle charging behavior optimization method based on deep reinforcement learning
WO2021143075A1 (en) * 2020-01-17 2021-07-22 南京东博智慧能源研究院有限公司 Demand response method taking space-time distribution of electric vehicle charging loads into consideration
CN113159578A (en) * 2021-04-22 2021-07-23 杭州电子科技大学 Charging optimization scheduling method of large-scale electric vehicle charging station based on reinforcement learning
CN113515884A (en) * 2021-04-19 2021-10-19 国网上海市电力公司 Distributed electric vehicle real-time optimization scheduling method, system, terminal and medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI687785B (en) * 2019-02-25 2020-03-11 華碩電腦股份有限公司 Method of returning to charging station
CN110570050A (en) * 2019-09-25 2019-12-13 国网浙江省电力有限公司经济技术研究院 Road-network-vehicle-related electric vehicle charging guiding method
WO2021143075A1 (en) * 2020-01-17 2021-07-22 南京东博智慧能源研究院有限公司 Demand response method taking space-time distribution of electric vehicle charging loads into consideration
CN111934335A (en) * 2020-08-18 2020-11-13 华北电力大学 Cluster electric vehicle charging behavior optimization method based on deep reinforcement learning
CN113515884A (en) * 2021-04-19 2021-10-19 国网上海市电力公司 Distributed electric vehicle real-time optimization scheduling method, system, terminal and medium
CN113159578A (en) * 2021-04-22 2021-07-23 杭州电子科技大学 Charging optimization scheduling method of large-scale electric vehicle charging station based on reinforcement learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
夏冬: "多信息融合下电动汽车充电路径规划", 电测与仪器, vol. 57, no. 22, 25 December 2019 (2019-12-25), pages 24 - 32 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024165229A1 (en) * 2023-02-08 2024-08-15 E.On Se Edge computing with ai at mains supply network interconnection points
CN116436019A (en) * 2023-04-12 2023-07-14 国网江苏省电力有限公司电力科学研究院 Multi-resource coordination optimization method, device and storage medium
CN116436019B (en) * 2023-04-12 2024-01-23 国网江苏省电力有限公司电力科学研究院 Multi-resource coordination optimization method, device and storage medium
CN118098000A (en) * 2024-04-24 2024-05-28 哈尔滨华鲤跃腾科技有限公司 Urban comprehensive management method based on artificial intelligence
CN118438918A (en) * 2024-05-09 2024-08-06 烟台开发区德联软件有限责任公司 Mobile energy storage device charging regulation and control method and device

Also Published As

Publication number Publication date
CN114444802B (en) 2024-06-04

Similar Documents

Publication Publication Date Title
CN114444802A (en) Electric vehicle charging guide optimization method based on graph neural network reinforcement learning
CN110570050B (en) Road-network-vehicle-related electric vehicle charging guiding method
Sun et al. A customized voltage control strategy for electric vehicles in distribution networks with reinforcement learning method
Luo et al. Optimal charging scheduling for large-scale EV (electric vehicle) deployment based on the interaction of the smart-grid and intelligent-transport systems
Xu et al. A soft actor-critic-based energy management strategy for electric vehicles with hybrid energy storage systems
Wang et al. Optimising route choices for the travelling and charging of battery electric vehicles by considering multiple objectives
Luo et al. Joint deployment of charging stations and photovoltaic power plants for electric vehicles
Panahi et al. Forecasting plug-in electric vehicles load profile using artificial neural networks
CN109523051A (en) A kind of electric car charging Real time optimal dispatch method
CN103241130A (en) Energy management method and system for electric bus charging and swap station
CN106875075A (en) A kind of electric automobile charging station points distributing method based on travel behaviour
CN113887858A (en) Charging station micro-grid system optimal scheduling method based on CNN-LSTM load prediction
CN103078152B (en) Intelligent charging method for centralized charging station
CN112330025B (en) Prediction method of space-time charging load for urban electric vehicle
Qian et al. Shadow-price DRL: A framework for online scheduling of shared autonomous EVs fleets
CN110276517A (en) A kind of electric automobile charging station site selecting method based on MOPSO algorithm
Kiaee Integration of electric vehicles in smart grid using deep reinforcement learning
Rahmani-Andebili et al. Optimal incentive plans for plug-in electric vehicles
CN111626492B (en) Fuzzy multi-target opportunity constraint planning method for electric vehicle charging network
Zhang et al. A safe reinforcement learning-based charging strategy for electric vehicles in residential microgrid
CN112381359A (en) Multi-critic reinforcement learning power economy scheduling method based on data mining
CN110197310B (en) Charging and battery replacing station optimal scheduling method based on load margin domain
Fan et al. Optimal shared mobility planning for electric vehicles in the distribution network
Pourvaziri et al. Planning of electric vehicle charging stations: An integrated deep learning and queueing theory approach
Chen et al. A Deep Reinforcement Learning-Based Charging Scheduling Approach with Augmented Lagrangian for Electric Vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant