CN111049125B - Electric vehicle intelligent access control method based on machine learning - Google Patents

Electric vehicle intelligent access control method based on machine learning Download PDF

Info

Publication number
CN111049125B
CN111049125B CN201910904347.1A CN201910904347A CN111049125B CN 111049125 B CN111049125 B CN 111049125B CN 201910904347 A CN201910904347 A CN 201910904347A CN 111049125 B CN111049125 B CN 111049125B
Authority
CN
China
Prior art keywords
charging
action
state
time
electric vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910904347.1A
Other languages
Chinese (zh)
Other versions
CN111049125A (en
Inventor
唐子昱
李紫昕
方明星
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Normal University
Original Assignee
Anhui Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui Normal University filed Critical Anhui Normal University
Priority to CN201910904347.1A priority Critical patent/CN111049125B/en
Publication of CN111049125A publication Critical patent/CN111049125A/en
Application granted granted Critical
Publication of CN111049125B publication Critical patent/CN111049125B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00—Circuit arrangements for ac mains or ac distribution networks
    • H02J3/008—Circuit arrangements for ac mains or ac distribution networks involving trading of energy or energy transmission rights
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/60—Monitoring or controlling charging stations
    • B60L53/64—Optimising energy costs, e.g. responding to electricity rates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Energy or water supply
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/60—Other road transportation technologies with climate change mitigation effect
    • Y02T10/70—Energy storage systems for electromobility, e.g. batteries
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/60—Other road transportation technologies with climate change mitigation effect
    • Y02T10/7072—Electromobility specific charging systems or methods for batteries, ultracapacitors, supercapacitors or double-layer capacitors
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
    • Y02T90/10—Technologies relating to charging of electric vehicles
    • Y02T90/12—Electric charging stations
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Health & Medical Sciences (AREA)
  • Power Engineering (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Water Supply & Treatment (AREA)
  • General Health & Medical Sciences (AREA)
  • Transportation (AREA)
  • Marketing (AREA)
  • Mechanical Engineering (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Charge And Discharge Circuits For Batteries Or The Like (AREA)
  • Electric Propulsion And Braking For Vehicles (AREA)

Abstract

The invention discloses an electric vehicle intelligent access control method based on machine learning, which comprises the following steps: 1, describing an access control process of a randomly arrived electric vehicle charging service request as an event-driven decision process; 2, taking the peak shaving electricity price of the power grid and the online service state of the charging pile as the joint state of the charging station service system; 3, taking the service request of the electric vehicle arriving at the charging station as an event, and when the event occurs, selecting whether the arriving electric vehicle is accessed to the charging station to provide the charging service as a system action according to the joint state of the charging station service system; and 4, performing online optimization on the intelligent access service system of the electric vehicle by adopting a Q learning machine learning algorithm. The intelligent access control method can effectively carry out intelligent access control on the electric vehicle to the charging station service system considering the peak regulation electricity price of the power grid, thereby improving the operation economy of the charging station and being adaptive to the peak regulation requirement of the power grid.

Description

Electric vehicle intelligent access control method based on machine learning
Technical Field
The invention belongs to the technical field of intelligent control and optimization, and particularly relates to an electric vehicle intelligent access control method based on machine learning.
Background
At present, China is the largest automobile consumption market in the world, automobile manufacturers have shifted research, development and production emphasis from automobiles powered by traditional energy sources to new energy automobiles, wherein electric automobiles are the mainstream of new energy automobile development in a long period of time, have huge consumption potential, and the market share is getting larger and larger. The charging pile is an important infrastructure for providing charging service for the electric automobile and is also an important link in the industrialization and commercialization processes of the electric automobile. With the rapid development of the electric automobile industry and the great improvement of the market reserves of electric automobiles, a charging station for performing centralized management and operation on a plurality of charging piles is an important business mode and service form in the future. In addition, the new energy permeability of wind power, photovoltaic and the like is improved, the intelligence and the adaptability of power production and service in the future are improved, and effective management and guidance of power utilization of power consumers are a trend. For example, each level of scheduling center can make an electric power peak shaving plan according to the source charge prediction data and issue the electric power peak shaving plan through real-time electricity price, so that electric power users such as electric vehicle charging stations are guided to reasonably use the electricity, and the automatic peak shaving and valley filling or peak shifting and valley filling at the user side is promoted.
The existing power grid electricity price adopts a very simple and fixed time-of-use electricity price mechanism, a power grid peak regulation electricity price plan is not dynamically formulated or adjusted according to the actual source charge prediction condition of a power grid, and a charging station service system does not dynamically and adaptively perform adaptive access control on a charging request of an electric vehicle according to the actual power grid peak regulation requirement. Therefore, under a real-time power grid peak regulation electricity price mechanism, how to carry out self-adaptive response on a charging service request of a randomly coming electric vehicle according to the real-time peak regulation electricity price of a power grid and the online service states of all charging piles in the station by the intelligent access service system of the electric vehicle of the charging station is to control whether the charging service request is accessed to the service, so that the operation economy of the charging station is improved, and the self-adaptive power grid peak regulation requirement is to be researched and solved.
Disclosure of Invention
The invention aims to solve the defects in the prior art, and provides an electric vehicle intelligent access control method based on machine learning so as to carry out effective online optimization control on a charging station service system in which an electric vehicle service request arrives at random, thereby improving the operation economy of the charging station and adapting to the peak regulation requirement of a power grid.
The invention adopts the following technical scheme for solving the technical problems:
the invention relates to an electric vehicle intelligent access control method based on machine learning, which is characterized in that the method is applied to a charging station service system which is provided with J charging piles and provides paid charging service for M electric vehicles which arrive randomly, each charging pile can meet the charging power requirements of the M electric vehicles, and one charging pile only provides charging service for one electric vehicle at a time;
recording the J charging piles as CS respectively1,CS2,…,CSj,…,CSJAnd the charging power requirement of the M electric vehicles is recorded as P1,P2,…,Pm,…,PMWherein CS isjIndicates the jth charging pile, PmRepresenting the charging power requirement of the mth electric vehicle;
let K be the maximum number of cycles per day and the corresponding total duration be T, and record the peak shaving electricity price of the power grid at any time T under the total duration T as PRtThen PRt∈ΦPR(ii) a Suppose that the peak regulation electricity price of the power grid is issued according to the dispatching instruction period and the instruction taukPeak-shaving power rate PR for the kthkWhen the electricity price is issued, the electricity price sequence of peak regulation is recorded as { (tau)k,PRk)|k＝0,1,2,…,K-1,τ00} where, PRk∈ΦPR，ΦPRIs a limited electricity price state space;
charging service price fixing PR of charging station service systemev；
Suppose that there is m at time ttThe electric vehicle randomly arrives at the electric station to apply for charging service, order the mthtThe current state of charge of the battery of the electric vehicle is
Figure GDA0002412578370000023
Then the m < th > istThe arrival event of the electric vehicle is recorded as
Figure GDA0002412578370000024
Recording the joint state of the J charging piles at the moment t as Ct＝(CS1(t),CS2(t),…,CSj(t),…,CSJ(t)), wherein
Figure GDA0002412578370000025
Representing the service state of the jth charging pile; m isj(t) indicates the jth charging pile CS at time tjClass of electric vehicle being serviced, if mj(t) ═ 0 denotes the jth charging pile CS at time tjNo vehicle access, if mj(t) is equal to {1,2, …, M } and represents the jth charging pile CS at the time tjCharging one electric vehicle in {1,2, …, M };
Figure GDA0002412578370000026
represents the jth charging pile CS at the time tjM < th > of being servedj(t) the current state of charge of the batteries of the electric vehicles;
m at ttArrival event of electric vehicle
Figure GDA0002412578370000022
The charging station service system status at the time of occurrence is recorded as st＝{Ct,PRtWill arrive at the event
Figure GDA0002412578370000027
Taking the occurrence time t as a decision time, and recording the event expansion state of the decision time as
Figure GDA0002412578370000021
Recording whether the charging station service system is connected to the electric vehicle or not and provides charging service as action a and recording the nth decision time TnIs anAnd a is anE.g., {0,1}, where "0" denotes denial of service, "1" denotes access to service, and D denotes a set of actions;
the intelligent access control method of the electric vehicle comprises the following steps:
step 1, defining and initializing nth decision time TnHas a movement search rate of ∈nAnd let 0 < epsilonn＜1；
Defining elements in a Q value table as discretized event expansion state-action pair learning values, and initializing the elements in the Q value table;
defining a current greedy control strategy table v as an action set formed by the maximum discretization event expansion state of each row in the Q value table and actions corresponding to action pair learning values;
step 2, initializing t to 0, and initializing n to 1; the current action exploration rate epsilonnAssign to epsilon1(ii) a Assigning the current greedy control strategy table v to the original strategy table v0；
Step 3, at the nth decision time T of the charging station service systemnArrival event
Figure GDA0002412578370000031
Occurrence and observation of the current united state s of the charging station service systemtEvent extended State
Figure GDA0002412578370000032
Let the nth decision time TnEvent extended state of
Figure GDA0002412578370000033
The corresponding discretization state in the Q-value table is recorded as
Figure GDA0002412578370000034
Let the nth decision time TnEvent extended state of
Figure GDA0002412578370000035
The action actually taken is recorded
Figure GDA0002412578370000036
Figure GDA0002412578370000037
At the nth decision time TnIf all charging piles are in service, i.e. { mj(t) ∈ {1,2, …, M } | J ═ 1,2, … J }, then let us
Figure GDA0002412578370000038
Otherwise, the current event is in an extended state
Figure GDA0002412578370000039
Then, extracting from the Q value table
Figure GDA00024125783700000310
Corresponding discretized state
Figure GDA00024125783700000311
Greedy action in the lower case
Figure GDA00024125783700000312
And with a probability of 1-epsilonnWill be provided with
Figure GDA00024125783700000313
Is assigned to
Figure GDA00024125783700000314
At the exploration rate εnGreedy removing actions in the action set D
Figure GDA00024125783700000315
Another action other than the search action is recorded as a search action
Figure GDA00024125783700000316
And assign a value to
Figure GDA00024125783700000317
The charging station service system takes action
Figure GDA00024125783700000318
Thereafter, the decision time T from the nth decision time is observednTransition to decision time T of n +1n+1Or system transfer sample track to time T
Figure GDA00024125783700000319
Wherein T is Tn，t′＝Tn+1T or T ═ T; when T ═ T, let
Figure GDA00024125783700000320
Step 4, observing and calculating the nth decision time T of the charging station service systemnCurrent state of
Figure GDA00024125783700000321
Take action
Figure GDA00024125783700000322
Transition to decision time T of n +1n+1Or to a state at a time T
Figure GDA00024125783700000323
Charging reward obtained during the state transition of
Figure GDA00024125783700000324
And 5, updating the Q value table by using a difference formula and a Q value updating formula shown in the formulas (1) and (2)
Figure GDA00024125783700000325
Corresponding discretized states
Figure GDA00024125783700000326
Take action
Figure GDA00024125783700000327
Discretized event extended state-action pair learned values of
Figure GDA00024125783700000328
And reassign to
Figure GDA00024125783700000329
Figure GDA00024125783700000330
Figure GDA00024125783700000331
In the formula (1), the reaction mixture is,
Figure GDA0002412578370000041
indicating a transition to the n +1 th decision time Tn+1Or to a state at a time T
Figure GDA0002412578370000042
Corresponding discretized states
Figure GDA0002412578370000043
Next if the discretized event of action a is taken, extend state-action pair learned value;
in equation (2), the operator ": "means that the value of the right formula is calculated first and then given to the left variable;
Figure GDA0002412578370000044
for the nth decision time TnDiscretized state of
Figure GDA0002412578370000045
Take action
Figure GDA0002412578370000046
The learning step length of (1);
step 6, selecting the action corresponding to the maximum discretization event expansion state-action pair learning value of each row in the updated Q value table and forming a current action set, and assigning the current action set to a current greedy control strategy v by taking the current action set as an updated greedy control strategy table; and for the exploration rate epsilonnPerforming a decay operation to obtain an updated exploration rate and assigning εn+1；
7, if T' is less than T, assigning n +1 to n, and returning to the step 3; otherwise, T' is represented as T, and step 8 is performed;
step 8, judging whether the control strategy table v is equal to v0And if the current charging service requests are equal to the M charging service requests, stopping updating and performing access control on the M charging service requests of the electric vehicles by using the current control strategy table v, otherwise, returning to the step 2 for execution.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a method for controlling a power grid peak shaving system, which takes a randomly arrived electric vehicle charging service request as an event, a system decides whether the arrived electric vehicle is accessed to a charging station to provide charging service when the event occurs according to an event expansion state formed by event occurrence time, a real-time state of a charging pile in the system, the current power grid peak shaving electricity price, the type of the arrived electric vehicle and an SOC state value of the electric vehicle, and takes the event occurrence time and the current power grid peak shaving electricity price as a part of the event expansion state, thereby being beneficial to reflecting the time sequence characteristic of the power grid peak shaving, leading the control strategy to be adaptive to the power grid peak shaving requirement, being more accordant with the actual situation and improving the feasibility of the method.
2. The method takes the peak-shaving electricity price of the power grid and the online service state of the charging pile as the joint state of the charging station service system; taking a randomly arrived electric vehicle charging service request as an event; combining the randomly generated events with the combined state of the charging station service system to form an event extended state; taking whether the arriving electric vehicle is accessed to a charging station to provide charging service as a system action; taking the moment when the electric vehicle charging service request randomly arrives as decision-making moment; the intelligent access control process of the electric vehicle at the charging station, in which the electric vehicle arrives randomly, is described as a discrete event-driven decision-making process, and corresponding action is taken according to the real-time event expansion state of the system; therefore, the problem of access control of electric vehicles at the charging station, which is caused by random arrival of electric vehicle service requests, is effectively solved, the system can reasonably select access actions through optimization, the operation economy of the charging station service system is improved, and the peak regulation requirement of a power grid can be self-adapted;
3. compared with a theoretical solving method, the intelligent charging station electric vehicle access control method does not need to carry out complete mathematical modeling on a control system, and particularly does not need to carry out accurate modeling on random characteristics in the system. According to the invention, a better control strategy can be obtained only by performing real-time online learning through the operation sample of the observation system. In addition, when the random parameters of the system change, an operator does not need to modify the algorithm, online learning can still be carried out according to the running process of the actual system, and a better intelligent access control strategy of the electric vehicle can be obtained in a self-adaptive manner;
4. the intelligent access control method for the electric vehicle is also suitable for different charging price time-interval situations and power grid peak regulation and non-periodic issuing situations.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
fig. 2 is a schematic diagram of a charging station service system according to the present invention.
Detailed Description
In this embodiment, as shown in fig. 2, an electric vehicle intelligent access control method based on machine learning is applied to a charging station service system composed of J charging piles 1, M electric vehicles 2 arriving at random, a power grid peak shaving electricity price plan 3, and an access control center 4, and each charging pile can adaptively meet the charging power requirements of the M electric vehicles;
recording the jth charging pile as CSjAnd only one electric vehicle is provided with charging service at a time; thereby recording J charging piles as CS1,CS2,…,CSj,…,CSJ，j＝1,2,…,J；
Recording the charging power requirement of the mth electric vehicle as PmKW, and total battery capacity of EmKWH, determined by the configuration of the electric vehicle itself; thereby recording the charging power requirement of M electric vehicles as P1,P2,…,Pm,…,PM，m＝1,2,…,M；
Let K be the maximum number of cycles per day and the corresponding total duration be T, and record the peak-shaving electricity price state of the power grid at any time T under the total duration T as PRtYuan/kilowatt-hour, and PRt∈ΦPR，ΦPRIs a limited electricity price state space; suppose that the peak regulation electricity price of the power grid is issued according to the dispatching instruction period and taukPeak-shaving power rate PR for the kthkThe time of delivery, the price is maintained to the next peak-shaving electricity price delivery time tauk+1To that end, i.e. PRt＝PRk，τk≤t＜τk+1K is 0,1,2, …, K-1 and τ00; the sequence of peak-shaving electrovalence is recorded as { (τ)k,PRk)|k＝0,1,2,…,K-1,τ0＝0}；
The charging station provides paid charging service, and the charging service price of the charging station is PRevYuan/kilowatt-hour;
setting the battery SOC at the time t as
Figure GDA0002412578370000051
M oftThe electric vehicle randomly arrives at the electric station to apply for charging service and is recorded as an arrival event
Figure GDA0002412578370000052
Recording the service state of the jth charging pile as
Figure GDA0002412578370000053
Thereby recording the joint state of J charging piles at the moment t as Ct＝(CS1(t),CS2(t),…,CSj(t),…,CSJ(t)); when t is assumed to be 0, all charging piles are empty; m isj(t) indicates the jth charging pile CS at time tjClass of electric vehicle being serviced, if mj(t) ═ 0 denotes the jth charging pile CS at time tjNo vehicle access, if mj(t) is equal to {1,2, …, M } and represents the jth charging pile CS at the time tjCharging one electric vehicle in {1,2, …, M };
Figure GDA0002412578370000061
represents the jth charging pile CS at the time tjM < th > of being servedj(t) battery SOC of electric vehicles;
will arrive at an event
Figure GDA0002412578370000062
The charging station service system status at the time of occurrence is recorded as st＝{Ct,PRtThe extension state of the note piece is
Figure GDA0002412578370000063
Note the nth event
Figure GDA0002412578370000064
The moment of occurrence is decision moment TnI.e. T ═ TnCorresponding grid price peak-to-peak hours
Figure GDA0002412578370000065
Namely, it is
Figure GDA0002412578370000066
Let τ beK＝T；
Discretizing the variation interval of the SOC of the electric vehicle battery by a small constant delta [ 01 ]]Then obtain
Figure GDA0002412578370000067
Corresponding discretized event extended state
Figure GDA0002412578370000068
Wherein the subscript "n" indicates the corresponding nth decision time TnA numerical or discretized value of; m isnIt is shown
Figure GDA0002412578370000069
Figure GDA00024125783700000610
Is CtThe corresponding discretized charging pile is in a combined state,
Figure GDA00024125783700000611
is CSj(t) corresponding discretization states, and
Figure GDA00024125783700000612
Figure GDA00024125783700000613
phi is a state space formed by all possible discretization event expansion states, and the total discretization event expansion state number of the system is recorded as S;
defining the system decision time as the arrival time of any electric vehicle, namely the event occurrence time;
taking whether the charging station service system is connected to the electric vehicle or not and providing the charging service as a control action a, and recording the nth decision time TnIs anAnd a is anE.g., {0,1}, where "0" denotes denial of service, "1" denotes access to service, and D denotes a set of actions; at any decision time TnIf m isj(t) ≠ 0, J ≠ 1,2, …, J, which means that all charging posts are busy, then an≡0；
Encoding all possible discretization event extension states to order
Figure GDA00024125783700000615
Represents the state of the spreading of the s-th discretization event, and
Figure GDA00024125783700000614
coding the spreading state of all possible discretization events with busy charging piles into the last discretization event and recording the state number as Sb；
At the nth decision time TnIf anIf the number of the electric vehicles reaches 1, the electric vehicles are connected to any idle charging pile and immediately charged; leaving the charging station immediately assuming one electric vehicle is full;
as shown in fig. 1, the intelligent access control method for the electric vehicle based on machine learning is performed according to the following steps:
step 1, defining and initializing nth decision time TnHas a movement search rate of ∈nAnd let 0 < epsilonn< 1, e.g. let εn＝0.8；
Defining elements in Q-value table as being offDiversifying event spread state-action pair learning values and initializing elements in the Q-value table, e.g., randomly initializing the value of each element or making it 0; the Q value table takes the discretization event expansion state of the system as the row of the Q value table and the access action of the system as the column of the Q value table, namely
Figure GDA0002412578370000071
Wherein Q value table last SbThe action corresponding to the row is fixed to '0';
defining a current greedy control strategy table v as an action set formed by the maximum discretization event expansion state of each row in a Q value table and actions corresponding to action pair learning values;
step 2, initializing variables t ═ 0 and n ═ 1; the current action exploration rate epsilonnAssign to epsilon1(ii) a Let original policy table v0＝v；
Step 3, at the nth decision time T of the charging station service systemnArrival event
Figure GDA0002412578370000072
Occurrence, observation of the current federated state s of the service systemtThe extension state of the note piece is
Figure GDA0002412578370000073
Let the nth decision time TnThe current event extended state of
Figure GDA0002412578370000074
The corresponding discretization state in the Q-value table is recorded as
Figure GDA0002412578370000075
Let the nth decision time TnCurrent event extended state of
Figure GDA0002412578370000076
The action actually taken is recorded
Figure GDA0002412578370000077
Figure GDA0002412578370000078
At the nth decision time TnIf all charging piles are in service, i.e. { mj(t) ∈ {1,2, …, M } | J ═ 1,2, … J }, then let us
Figure GDA0002412578370000079
Otherwise, the current event is in an extended state
Figure GDA00024125783700000710
Next, it is extracted from the Q value table
Figure GDA00024125783700000711
Corresponding state
Figure GDA00024125783700000712
Greedy action in the lower case
Figure GDA00024125783700000713
And with a probability of 1-epsilonnWill be provided with
Figure GDA00024125783700000714
Is assigned to
Figure GDA00024125783700000715
And at a search rate of ∈nGreedy removal actions in action set D
Figure GDA00024125783700000716
Another action than the search action
Figure GDA00024125783700000717
Is assigned to
Figure GDA00024125783700000718
Charging station service system takes action
Figure GDA00024125783700000719
Thereafter, the decision time T from the nth decision time is observednTransition to decision time T of n +1n+1Or to the transfer sample track at time T
Figure GDA00024125783700000720
Wherein T is Tn,t′＝Tn+1T or T ═ T; when T ═ T, it is assumed that
Figure GDA00024125783700000721
Step 4, calculating the nth decision time T of the charging station service system by using the formula (1)nCurrent state of
Figure GDA0002412578370000081
Take action
Figure GDA0002412578370000082
Then, the decision time T is shifted to the n +1 th decision time Tn+1Or to a state at a time T
Figure GDA0002412578370000083
Accumulated reward generated during the state transition of (1)
Figure GDA0002412578370000084
Figure GDA0002412578370000085
In formula (1), m is definedjSgn (m) when (t) is 0j(t))＝0，mjSgn (m) with (t) > 0j(t)) ═ 1; let T ═ min { T }n+1,T}；
Figure GDA0002412578370000086
Denotes the m-thj(t) charging power requirements of electric vehicles;
and 5, updating by using the difference formula and the Q value shown in the formula (2) and the formula (3)Formula, update in Q value table
Figure GDA0002412578370000087
Corresponding discretized states
Figure GDA0002412578370000088
Take action
Figure GDA0002412578370000089
Discretized event extended state-action pair learned values of
Figure GDA00024125783700000810
Obtaining updated learning value and assigning to
Figure GDA00024125783700000811
Figure GDA00024125783700000812
Figure GDA00024125783700000813
In the formula (2), the reaction mixture is,
Figure GDA00024125783700000814
indicating a transition to the n +1 th decision time TnOr to a state at a time T
Figure GDA00024125783700000815
Corresponding discretized states
Figure GDA00024125783700000816
Discretized event expansion state-action pair learned value of action a is taken;
in equation (3), the operator ": "means that the value of the right formula is calculated first and then given to the left variable;
Figure GDA00024125783700000817
expanding state for current discretization event at nth decision time
Figure GDA00024125783700000818
Take action
Figure GDA00024125783700000819
The learning step length of (1);
step 6, selecting the action corresponding to the maximum discretization event expansion state-action pair learning value of each row in the updated Q value table and forming a current action set, and taking the current action set as an updated greedy control strategy table and assigning the updated greedy control strategy table to a current greedy control strategy v; and for the exploration rate epsilonnPerforming a decay operation to obtain an updated exploration rate and assigning εn+1；
And 7, if T' ═ Tn+1If the value is less than T, assigning n +1 to n, and returning to the step 3; otherwise, indicating that T ═ T, step 8 is performed;
step 8, judging whether the control strategy table v is equal to v0And if the request is equal to the request, stopping updating and performing access control on the random charging service requests of the M types of electric vehicles by using the final control strategy table, otherwise, returning to the step 2 for execution.

Claims (1)

1. An electric vehicle intelligent access control method based on machine learning is characterized in that the method is applied to a charging station service system which is provided with J charging piles and provides paid charging service for M electric vehicles which arrive randomly, each charging pile can meet the charging power requirements of the M electric vehicles, and one charging pile only provides charging service for one electric vehicle at a time;
recording the J charging piles as CS respectively1,CS2,…,CSj,…,CSJAnd the charging power requirement of the M electric vehicles is recorded as P1,P2,…,Pm,…,PMWherein CS isjIndicates the jth charging pile, PmRepresenting the charging power requirement of the mth electric vehicle;
let K be the maximum number of cycles per day and the corresponding total duration be T, willThe peak regulation electricity price of the power grid at any time T under the total time length T is recorded as PRtThen PRt∈ΦPR(ii) a Suppose that the peak regulation electricity price of the power grid is issued according to the dispatching instruction period and the instruction taukPeak-shaving power rate PR for the kthkWhen the electricity price is issued, the electricity price sequence of peak regulation is recorded as { (tau)k,PRk)|k＝0,1,2,…,K-1,τ00} where, PRk∈ΦPR，ΦPRIs a limited electricity price state space;
charging service price fixing PR of charging station service systemev；
Suppose that there is m at time ttThe electric vehicle randomly arrives at the electric station to apply for charging service, order the mthtThe current state of charge of the battery of the electric vehicle is
Figure FDA0002916817390000011
Then the m < th > istThe arrival event of the electric vehicle is recorded as
Figure FDA0002916817390000012
Recording the joint state of the J charging piles at the moment t as Ct＝(CS1(t),CS2(t),…,CSj(t),…,CSJ(t)), wherein
Figure FDA0002916817390000013
Representing the service state of the jth charging pile; m isj(t) indicates the jth charging pile CS at time tjClass of electric vehicle being serviced, if mj(t) ═ 0 denotes the jth charging pile CS at time tjNo vehicle access, if mj(t) is equal to {1,2, …, M } and represents the jth charging pile CS at the time tjCharging one electric vehicle in {1,2, …, M };
Figure FDA0002916817390000014
represents the jth charging pile CS at the time tjM < th > of being servedj(t) the current state of charge of the batteries of the electric vehicles;
m at ttArrival event of electric vehicle
Figure FDA0002916817390000015
The charging station service system status at the time of occurrence is recorded as st＝{Ct,PRtWill arrive at the event
Figure FDA0002916817390000016
Taking the occurrence time t as a decision time, and recording the event expansion state of the decision time as
Figure FDA0002916817390000017
Recording whether the charging station service system is connected to the electric vehicle or not and provides charging service as action a and recording the nth decision time TnIs anAnd a is anE.g., {0,1}, where "0" denotes denial of service, "1" denotes access to service, and D denotes a set of actions;
the intelligent access control method of the electric vehicle comprises the following steps:
step 1, defining and initializing nth decision time TnHas a movement search rate of ∈nAnd let 0 < epsilonn＜1；
Defining elements in a Q value table as discretization event expansion states-action pair learning values, and initializing the elements in the Q value table, namely randomly initializing the value of each element or making the value of each element be 0; the Q value table takes the discretization extended event state of the system as the row of the Q value table and the access action of the system as the column of the Q value table, namely
Figure FDA0002916817390000021
Wherein Q value table last SbThe action corresponding to the row is fixed to '0';
defining a current greedy control strategy table v as an action set formed by the maximum discretization event expansion state of each row in the Q value table and actions corresponding to action pair learning values;
step 2, initialChanging t to 0 and n to 1; the current action exploration rate epsilonnAssign to epsilon1(ii) a Assigning the current greedy control strategy table v to the original strategy table v0；
Step 3, at the nth decision time T of the charging station service systemnArrival event
Figure FDA0002916817390000022
Occurrence and observation of the current united state s of the charging station service systemtEvent extended State
Figure FDA0002916817390000023
Let the nth decision time TnEvent extended state of
Figure FDA0002916817390000024
The corresponding discretization state in the Q-value table is recorded as
Figure FDA0002916817390000025
Let the nth decision time TnEvent extended state of
Figure FDA0002916817390000026
The action actually taken is recorded
Figure FDA0002916817390000027
At the nth decision time TnIf all charging piles are in service, i.e. { mj(t) ∈ {1,2, …, M } | J ═ 1,2, … J }, then let us
Figure FDA0002916817390000028
Otherwise, the current event is in an extended state
Figure FDA0002916817390000029
Then, extracting from the Q value table
Figure FDA00029168173900000210
Corresponding discretized state
Figure FDA00029168173900000211
Greedy action in the lower case
Figure FDA00029168173900000212
And with a probability of 1-epsilonnWill be provided with
Figure FDA00029168173900000213
Is assigned to
Figure FDA00029168173900000214
At the exploration rate εnGreedy removing actions in the action set D
Figure FDA00029168173900000215
Another action other than the search action is recorded as a search action
Figure FDA00029168173900000216
And assign a value to
Figure FDA00029168173900000217
The charging station service system takes action
Figure FDA00029168173900000218
Thereafter, the decision time T from the nth decision time is observednTransition to decision time T of n +1n+1Or system transfer sample track to time T
Figure FDA00029168173900000219
Wherein T is Tn，t′＝Tn+1T or T ═ T; when T ═ T, let
Figure FDA00029168173900000220
Step 4, observing and calculating the nth decision time T of the charging station service systemnCurrent state of
Figure FDA00029168173900000221
Take action
Figure FDA00029168173900000222
Transition to decision time T of n +1n+1Or to a state at a time T
Figure FDA0002916817390000031
Charging reward obtained during the state transition of
Figure FDA0002916817390000032
And 5, updating the Q value table by using a difference formula and a Q value updating formula shown in the formulas (1) and (2)
Figure FDA0002916817390000033
Corresponding discretized states
Figure FDA0002916817390000034
Take action
Figure FDA0002916817390000035
Discretized event extended state-action pair learned values of
Figure FDA0002916817390000036
And reassign to
Figure FDA0002916817390000037
Figure FDA0002916817390000038
Figure FDA0002916817390000039
In the formula (1), the reaction mixture is,
Figure FDA00029168173900000310
indicating a transition to the n +1 th decision time Tn+1Or to a state at a time T
Figure FDA00029168173900000311
Corresponding discretized states
Figure FDA00029168173900000312
Next if the discretized event of action a is taken, extend state-action pair learned value;
in equation (2), the operator ": "means that the value of the right formula is calculated first and then given to the left variable;
Figure FDA00029168173900000313
for the nth decision time TnDiscretized state of
Figure FDA00029168173900000314
Take action
Figure FDA00029168173900000315
The learning step length of (1);
step 6, selecting the action corresponding to the maximum discretization event expansion state-action pair learning value of each row in the updated Q value table and forming a current action set, and assigning the current action set to a current greedy control strategy v by taking the current action set as an updated greedy control strategy table; and for the exploration rate epsilonnPerforming a decay operation to obtain an updated exploration rate and assigning εn+1；
7, if T' is less than T, assigning n +1 to n, and returning to the step 3; otherwise, T' is represented as T, and step 8 is performed;
step 8, judging whether the control strategy table v is equal to v0And if the current charging service requests are equal to the M electric vehicles, stopping updating and performing access control on the random charging service requests of the M electric vehicles by using the current control strategy table v, otherwise, returning to the step 2 for execution.
CN201910904347.1A 2019-09-24 2019-09-24 Electric vehicle intelligent access control method based on machine learning Active CN111049125B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910904347.1A CN111049125B (en) 2019-09-24 2019-09-24 Electric vehicle intelligent access control method based on machine learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910904347.1A CN111049125B (en) 2019-09-24 2019-09-24 Electric vehicle intelligent access control method based on machine learning

Publications (2)

Publication Number Publication Date
CN111049125A CN111049125A (en) 2020-04-21
CN111049125B true CN111049125B (en) 2021-07-30

Family

ID=70232490

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910904347.1A Active CN111049125B (en) 2019-09-24 2019-09-24 Electric vehicle intelligent access control method based on machine learning

Country Status (1)

Country Link
CN (1) CN111049125B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107719170A (en) * 2017-10-31 2018-02-23 云南电网有限责任公司 Electric automobile charging pile timing charging method and device for power distribution network peak regulation
CN108944516A (en) * 2018-07-19 2018-12-07 华北电力大学 It is a kind of to be charged incentive mechanism based on block chain and the electric car of machine learning
CN109130935A (en) * 2018-08-30 2019-01-04 广东电网有限责任公司 A kind of electric car intelligent charge monitoring system
CN110190597A (en) * 2019-05-31 2019-08-30 华北电力大学 A kind of distributed power management system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11545830B2 (en) * 2017-01-18 2023-01-03 Board Of Regents, The University Of Texas System Systems and methods of hierarchical forecasting of solar photovoltaic energy production
CN106786977B (en) * 2017-01-22 2019-12-13 中南大学 Charging scheduling method of electric vehicle charging station
US11135937B2 (en) * 2018-04-06 2021-10-05 Cisco Technology, Inc. Vehicle charging leveraging telecommunication infrastructure
CN109409388B (en) * 2018-11-07 2021-08-27 安徽师范大学 Dual-mode deep learning descriptor construction method based on graphic primitives
CN109808541B (en) * 2019-01-17 2020-11-10 北京理工新源信息科技有限公司 Electric vehicle charging method and system
CN110211671B (en) * 2019-05-28 2021-03-16 安徽师范大学 Thresholding method based on weight distribution

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107719170A (en) * 2017-10-31 2018-02-23 云南电网有限责任公司 Electric automobile charging pile timing charging method and device for power distribution network peak regulation
CN108944516A (en) * 2018-07-19 2018-12-07 华北电力大学 It is a kind of to be charged incentive mechanism based on block chain and the electric car of machine learning
CN109130935A (en) * 2018-08-30 2019-01-04 广东电网有限责任公司 A kind of electric car intelligent charge monitoring system
CN110190597A (en) * 2019-05-31 2019-08-30 华北电力大学 A kind of distributed power management system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
机器学习在能源与电力系统领域的应用和展望;程乐峰,余涛,张孝顺;《电力系统自动化》;20190110;第43卷(第1期);15-24页 *

Also Published As

Publication number Publication date
CN111049125A (en) 2020-04-21

Similar Documents

Publication Publication Date Title
Liang et al. Mobility-aware charging scheduling for shared on-demand electric vehicle fleet using deep reinforcement learning
Abdullah et al. Reinforcement learning based EV charging management systems–a review
CN111934335B (en) Cluster electric vehicle charging behavior optimization method based on deep reinforcement learning
Ahmed et al. The role of artificial intelligence in the mass adoption of electric vehicles
CN102256369B (en) Task scheduling method for wireless sensor grid based on energy and communication overhead
CN116001624A (en) Ordered charging method for one-pile multi-connected electric automobile based on deep reinforcement learning
CN112865190A (en) Optimal scheduling method and system for photovoltaic and charging demand-based optical storage charging station
CN104217258A (en) Method for power load condition density prediction
CN113326994A (en) Virtual power plant energy collaborative optimization method considering source load storage interaction
Mbuwir et al. A hybrid policy gradient and rule-based control framework for electric vehicle charging
Wang et al. Event-based electric vehicle scheduling considering random user behaviors
CN112757922A (en) Hybrid power energy management method and system for vehicle fuel cell
CN111313449B (en) Cluster electric vehicle power optimization management method based on machine learning
Wu et al. Electric vehicle charging scheduling considering infrastructure constraints
CN114282821A (en) Scheduling method, system and equipment for sharing electric automobile
CN115587645A (en) Electric vehicle charging management method and system considering charging behavior randomness
CN114692965A (en) Electric vehicle cluster schedulable potential prediction method and device based on digital-analog hybrid driving
CN111284347A (en) State clustering coding method in charging station vehicle access control
Jin et al. Optimal priority rule enhanced deep reinforcement learning for charging scheduling in an electric vehicle battery swapping station
Ni et al. Mobility and energy management in electric vehicle based mobility-on-demand systems: Models and solutions
Li et al. Data-driven bi-level predictive energy management strategy for fuel cell buses with algorithmics fusion
CN110991931B (en) Charging station cooperative optimization control method based on double-center Q learning
CN111049125B (en) Electric vehicle intelligent access control method based on machine learning
CN114619907A (en) Coordinated charging method and coordinated charging system based on distributed deep reinforcement learning
CN116843500A (en) Charging station planning method, neural network model training method, device and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant