WO2024078728A1 - Gestion intelligente de puissance de réseau pour charge de véhicule de flotte - Google Patents

Gestion intelligente de puissance de réseau pour charge de véhicule de flotte Download PDF

Info

Publication number
WO2024078728A1
WO2024078728A1 PCT/EP2022/078734 EP2022078734W WO2024078728A1 WO 2024078728 A1 WO2024078728 A1 WO 2024078728A1 EP 2022078734 W EP2022078734 W EP 2022078734W WO 2024078728 A1 WO2024078728 A1 WO 2024078728A1
Authority
WO
WIPO (PCT)
Prior art keywords
charging
battery
vehicle
power
energy
Prior art date
Application number
PCT/EP2022/078734
Other languages
English (en)
Inventor
Jonas Hellgren
Original Assignee
Volvo Truck Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Volvo Truck Corporation filed Critical Volvo Truck Corporation
Priority to PCT/EP2022/078734 priority Critical patent/WO2024078728A1/fr
Publication of WO2024078728A1 publication Critical patent/WO2024078728A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J7/00—Circuit arrangements for charging or depolarising batteries or for supplying loads from batteries
    • H02J7/0013—Circuit arrangements for charging or depolarising batteries or for supplying loads from batteries acting upon several batteries simultaneously or sequentially
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/60—Monitoring or controlling charging stations
    • B60L53/63—Monitoring or controlling charging stations in response to network capacity
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/60—Monitoring or controlling charging stations
    • B60L53/64—Optimising energy costs, e.g. responding to electricity rates
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/60—Monitoring or controlling charging stations
    • B60L53/65—Monitoring or controlling charging stations involving identification of vehicles or their battery types
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/60—Monitoring or controlling charging stations
    • B60L53/67—Controlling two or more charging stations
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/60—Monitoring or controlling charging stations
    • B60L53/68—Off-site monitoring or control, e.g. remote control
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L58/00—Methods or circuit arrangements for monitoring or controlling batteries or fuel cells, specially adapted for electric vehicles
    • B60L58/10—Methods or circuit arrangements for monitoring or controlling batteries or fuel cells, specially adapted for electric vehicles for monitoring or controlling batteries
    • B60L58/12—Methods or circuit arrangements for monitoring or controlling batteries or fuel cells, specially adapted for electric vehicles for monitoring or controlling batteries responding to state of charge [SoC]
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00—Circuit arrangements for ac mains or ac distribution networks
    • H02J3/008—Circuit arrangements for ac mains or ac distribution networks involving trading of energy or energy transmission rights
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J7/00—Circuit arrangements for charging or depolarising batteries or for supplying loads from batteries
    • H02J7/007—Regulation of charging or discharging current or voltage
    • H02J7/0071—Regulation of charging or discharging current or voltage with a programmable schedule
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L2240/00—Control parameters of input or output; Target parameters
    • B60L2240/80—Time limits
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/60—Other road transportation technologies with climate change mitigation effect
    • Y02T10/70—Energy storage systems for electromobility, e.g. batteries
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/60—Other road transportation technologies with climate change mitigation effect
    • Y02T10/7072—Electromobility specific charging systems or methods for batteries, ultracapacitors, supercapacitors or double-layer capacitors

Definitions

  • the present disclosure relates to a method of power management for charging a plurality of electric vehicles, for example, a fleet of electric vehicles, and to various related aspects.
  • Electric vehicles are periodically connected to the electric grid or other charging sources, which may themselves draw power from the grid, in order to recharge their on-board battery systems.
  • Battery charging systems found on electric vehicles may store anything from a few kWs to more than a 100kWs of electrical energy depending on their intended range and what other systems may draw from the on-board vehicle battery.
  • certain types of vehicles may be connected to vehicle accessories.
  • Such accessories may draw from the vehicle battery system or have their own on-board battery systems which may be charged directly or via a connected vehicle. Such accessories may accordingly further increase the demand for power when recharging an electric vehicle to which they are electrically connected.
  • charging an on-board vehicle battery charging takes several hours.
  • the electric energy consumed from the grid should ideally be managed so that the charging is managed in a way that avoids excessive power demands from the grid and ideally also reduces the cost per electrical energy unit used in the battery charging process. This is particular important where a number of vehicles operating on a site may require charging, especially if they are large heavy vehicles which may have substantial battery storage capacity on-board to recharge.
  • the price per kW/h unit may vary according to conditions which affect the output of electricity from these systems. For example, the price may be less if the wind is blowing more strongly or be higher if there is no wind, less if the waves are the right height, but more or less if they are not, and also depend on the prevailing amount of solar flux. Such adaptations may be in real-time or delayed.
  • Some embodiments of the disclosed technology provide a method of controlling in an adaptive and optimal manner when a plurality of electric vehicles should be charged or not.
  • a related charging algorithm for determining when an electric vehicle should be charged or not charged using a charging model is set out in the inventor’s related International Patent Application PCT/EP2022/078719, the contents of which are hereby incorporated in their entirety.
  • the disclosed technology relates to a method of charging a fleet of electric vehicles which uses a self-learning charging model to determine when to charge individual vehicles in the fleet of electric vehicles.
  • the charging models may be implemented using reinforcement learning algorithms which are configured to determine when an action to charge one or more vehicles in the fleet should be taken.
  • the disclosed technology relates to a self-learning method of grid power management for charging an electric vehicle which seeks to control in an adaptive and optimal manner when one or more or all electric vehicles of a plurality of electric vehicles should be charged or not charged.
  • the method is particularly useful for electric vehicles which are automated in some way, for example, they may be autonomous or semi-autonomous and/or remote-controlled.
  • the disclosed invention will be described mainly with respect to vehicles, however, such vehicles may include heavy-duty vehicles, such as semi-trailer vehicles and trucks as well as other types of vehicles such as cars.
  • the disclosed technology seeks to mitigate, obviate, alleviate, or eliminate various issues known in the art which affect how to manage power consumption when charging an electric vehicle to avoid drawing power at times where power availability is lower due to demand. For example, in times of peak demand, to deter some customers from using electrical power distributed from a power grid, tariffs set for power consumption from the grid may be punitive. It is therefore helpful if power can be managed to avoid such periods of peak demand.
  • Determining when to charge an electric vehicles, EVs, in a residential charging environment may also be complicated by the maximum amp of the residential supply and the amount of power it will take to charge the electric vehicle as other devices such as a dishwasher or washing machine etc., may also need to draw power at the same time.
  • RL does not have to iterate for every time step as learning can be done completely offline and optimal solutions saved in a lookup table, from which optimal control actions can be retrieved almost instantaneously.
  • a first aspect of the disclosed technology comprises a computer-implemented method of power management for charging a plurality of batteries of a fleet of vehicles, each vehicle comprising at least one battery, the method comprising: determining a set of variables for each vehicle in the fleet, the set of variables including a current battery state of energy variable and a target battery state of energy at the end of a charging period; and determining if power should be provided to charge one or more vehicles in the fleet via or using at least one external power source based on output from a self-learning charging model configured to map a set of variables input to the model to an output comprising an action to charge the vehicle battery or not to charge the vehicle, based on one or more charging constraints for the fleet of vehicles.
  • the model are configured to take into account the one or more charging constraints when mapping an input set of variables to an output from the model.
  • the output may be stored output generated by previously training the self-learning model in some embodiments.
  • the power management method may enable a fleet of vehicles to be intelligently charged in a way that reduces electricity demand on a local power grid during peak hours.
  • a charging constraint for the fleet of vehicles comprises a cost constraint for charging the fleet of vehicles.
  • a charging constraint for the fleet of vehicles comprises a power constraint representing a power limit for the amount of power that can be transmitted to the fleet of vehicles at any given point in time.
  • the charging model is configured to minimize a standard deviation of battery state of energy levels across the fleet of vehicles.
  • the charging policy is configured to minimize a total deviation from a target battery state of energy level across the fleet of vehicles.
  • the current battery state of energy is determined when that vehicle is connected to an external power supply and the method further comprises, if that vehicle is determined to receive power using the charging policy, configuring the external power supply to provide power to charge the vehicle.
  • configuring the external power supply to provide power to charge that vehicle comprises actuating a soft switch or relay.
  • the charging model policy is configured to determine, for each of a plurality of charging periods, electricity cost and probable risk of a battery state of energy not meeting a target charged battery energy state at the end of each charging period.
  • the set of variables include a current time of day, a charging tariff for the current time of day, the target charged battery energy state and the current energy state of the battery.
  • using the self-learning model includes performing, for each vehicle of the fleet of vehicles, a method for charging an electric vehicle at a charging power source, wherein the method comprises: detecting a current state of energy of a battery of a vehicle, determining a target charged state of energy of the battery at a future point in time, and determining a point in time when power should be provided to charge the vehicle from the charging power source based on the detected current state of energy of the battery and the determined target charged state of energy of the battery so that the future point in time is a predicted end of a charging time period.
  • the future point in time may be determined using a vehicle battery charging model configured to determine the target charged battery energy state based on a predicted user demand for power from the charged vehicle at the end of the charging period and at least one charging constraint for the charging power source.
  • the method further comprises training the vehicle battery charging model to: map input to the model comprising a set of power load variables of the charging power source to an output representing a charging action to charge the battery or a non-charging action to not charge the battery by rewarding for long-term and short-term consequences of the charging or non-charging actions.
  • the method further comprises: when the battery is determined to start charging at the future point in time, based on the determined current state of energy of the battery and a determined charging action of the dynamic vehicle battery charging model, configuring the external charging power supply to automatically provide power to charge the vehicle by actuating a soft switch.
  • the method further comprises, responsive to determining when power should start to be provided to charge the vehicle to achieve the predicted target charged battery state by the determined future time, causing an alert to be provided to a user to connect the vehicle to a power supply if the vehicle is not already connected at or in advance of the determined start time.
  • At least one charging constraint is a charge or not condition based on an objective to minimize long term energy costs.
  • the set of power load variables of the external power source includes a power load variable of an external power supply to the charging power source, wherein the external power supply is connected to one or more other power-consuming devices consuming power in a charging period.
  • the charging constraint comprises a maximum available power constraint for the external power supply to provide power to the charging power supply.
  • the one or more power-consuming devices include one or more devices which are not battery operated.
  • the charging model is trained to determine, for each of a plurality of charging periods, electricity cost and a probable risk of a battery state of energy not meeting a target charged battery energy state at the end of that charging period.
  • the method further comprises varying the amount of charging power used by the charging power supply to charge the vehicle battery based on the current vehicle battery state of energy.
  • the method further comprises: obtaining data generated by implementing the dynamic vehicle battery charging model at the charging power supply, using the gathered data as a training data set for training the dynamic battery vehicle charging model, wherein the gathered data comprises, for each charging period, data representing the start time, the end time, the initial battery charge state, the end battery charge set, the power consumed charging the battery and the cost of the power consumed charging the battery to the charging model, and re- training the trained dynamic battery vehicle charging model by inputting the gathered data into the trained model.
  • the self-learning charging model is trained using reinforcement learning for use in a method according to the first aspect, the trained model being configured to determine when to charge each vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods, the method comprising: inputting training data to the charging policy model; and iteratively processing the input training data to provide iterative output representing a charging cost, wherein each iteration comprises: comparing a charging cost with a target charging cost or with a charging cost from a previous iteration; and maximising cumulative rewards for reducing the charging cost to achieve the target battery charge state.
  • each reward, r, of the maximised cumulative rewards can be represented for each charging period as either: a charge power consumption per unit time multiplied by a duration of time over which charge power is consumed multiplied by the energy consumption cost per unit of power in the case where the end of the charging period is reached and the target battery energy state is reached, or, if the target battery energy state is not reached at the end of the charging period, a total energy consumption cost for the duration of time the battery was charged plus a penalty term comprising the energy deficiency at the end of the charging period multiplies by a penalty constant.
  • each reward r of the maximised cumulative rewards is determined using a reward function based on a charge power consumption per unit time, a duration of time over which charge power is consumed, a battery energy deficiency comprising a difference between a target state of battery energy and a current state of battery energy, an energy consumption cost, and wherein the reward function indicates if an end of a charging episode is reached or not, and includes a penalty term defining how a violation from a target state of energy is to be penalised.
  • Another, second, aspect of the disclosed technology comprises an apparatus comprising a computer-program product comprising a set of instructions stored in a memory and comprising one or more modules or circuitry which, when loaded from the memory for execution by one or more processors or processing circuitry of the apparatus cause the apparatus to implement a method according to the first aspect or any combination of one or more of the above embodiments disclosed herein.
  • Another, third, aspect of the disclosed technology comprises a fleet vehicle charging management system, the fleet comprising a plurality of electric vehicles 100a,b,c, wherein the charging management system comprises: an apparatus according to the second aspect, a plurality of soft-switches, a plurality of battery chargers configured to be connected to a grid power supply, where each respective one of the battery chargers is configured to provide a charging power supply to a connected electric vehicle of the plurality of electric vehicles via a respective one of the soft- switches based on a self-learning battery charging model, for example, a self-learning battery charging model as disclosed herein, wherein the battery charging model is configured to minimise a financial cost and/or a power demand of implementing a connection policy control variable for charging the vehicle fleet.
  • the charging management system comprises: an apparatus according to the second aspect, a plurality of soft-switches, a plurality of battery chargers configured to be connected to a grid power supply, where each respective one of the battery chargers is configured to provide a charging power supply to a
  • Example of the grid power supply may comprise, for example, a site or premises distribution system, or a local distribution transformer source, or a distribution sub-station.
  • the plurality of vehicles accordingly may, for example, be connected to battery chargers which are powered via the same site or premises distribution system, or via the same local distribution transformer source, or via the same distribution sub-station.
  • a power management scheme is employed such as that of the disclosed technology, it is possible to draw too much power from grid at the same time. This may lead to shortages in supply and/or power out-ages which may not be limited to just the battery charging apparatus, but which could also affect other devices and consumers connected to the same distribution equipment of the local power grid.
  • the apparatus implements a battery charging model policy which selectively determines which of the plurality of connected electric vehicles is to be charged based on a SoE deviation from the average SoE of the vehicle fleet, and if a vehicle is determined to be charged, which determines a charging period for charging that vehicle’s battery system.
  • the charging period for each battery may be individually started by the apparatus causing a respective one of the soft-switches to be actuated so as to control a supply of electrical power to start charging the battery system of the connected electric vehicle.
  • the battery charging module is configured to minimise a connection policy control variable min cost ( u ) so that ⁇ ⁇ .. ⁇ P ⁇ ⁇ (u) ⁇ P ⁇ ⁇ ⁇ , where n is the number of vehicles in the fleet, Pch i is the Power consumed to charge the ith vehicle in the fleet, and Pch max fleet is the maximum power allowed to charge the fleet of vehicles.
  • at least one electric vehicle is a heavy duty electric vehicle.
  • at least one or more or all of the electric vehicles comprise an autonomous or semi-autonomous or remote controlled electric vehicle.
  • Another, fourth, aspect of the disclosed technology comprises a computer-implemented method for charging an electric vehicle at a charging power source, the method comprising: detecting a current state of energy of a battery of a vehicle, determining a target charged state of energy of the battery at a future point in time, and determining a point in time when power should be provided to charge the vehicle from the charging power source based on the detected current state of energy of the battery and the determined target charged state of energy of the battery so that the future point in time is a predicted end of a charging time period, wherein the future point in time is determined using a vehicle battery charging model configured to determine the target charged battery energy state based on a predicted user demand for power from the charged vehicle at the end of the charging period and at least one charging constraint for the charging power source.
  • the at least one charging constraint may include a charging constraint derived from the power demands for recharging batteries of or more other electric vehicles, for example, other electric vehicles whose battery chargers are connected to the same local electrical grid power distribution sub-station as those of the electric vehicle.
  • the method according to the fourth aspect includes a constraint for charging a plurality of electric vehicles including the electric vehicle. The method may include training the vehicle battery charging model to map input to the model comprising a set of power load variables of the charging power source to an output representing a charging action to charge the battery or a non-charging action to not charge the battery by rewarding for long-term and short- term consequences of the charging or non-charging actions.
  • the fourth aspect provides a computer-implemented method of power management for charging an electric vehicle of a fleet of electric vehicles, the method comprising determining a current state of energy of a battery of a vehicle determining a target desired charged state of energy of the battery at the end of a charging period, determining if power should be provided to charge the vehicle from a charging power source using a charging model for the power source based on the current state of energy of the battery and the determined target charged state of energy of the battery, wherein the dynamic charging model is trained to map input comprising a set of power load variables of the charging power source to an output comprising an action to charge the vehicle battery or not, wherein the model is trained using rewards for long-term consequences of the action and rewards for short-term consequences of the action.
  • the trained charging model determines a target charged battery energy state for each vehicle of a plurality of vehicles based on a predicted user demand for power from the charging power source at a predicted end of the charging period based on a cost of charging constraint for each vehicle and/or the fleet of vehicles.
  • the disclosed method may be implemented using a soft-switch so that a battery may be connected but only charged a subsequent point in time and still achieve a desired end state of energy.
  • the battery is determined to start charging at the future point in time, based on the determined current state of energy of the battery and a determined charging action of the dynamic vehicle battery charging model, configuring the external charging power supply to automatically provide power to charge the vehicle by actuating a soft switch or relay (S208).
  • the method does not need to perform a forecast for a supply load profile and day-ahead tariff.
  • the method may learn over time the optimal time to recharge a vehicle based on its use history and charging history so that its down-time for recharging is minimised.
  • the method has the potential to improve the overall use of an electric vehicle by improving the likelihood that it will have a desired battery state of energy for a predicted desired use at a particular point in time.
  • the technology may also improve the charging performance of the battery of an electric vehicle as the soft-switch can also turn off power to prevent over-charging.
  • the soft-switch may disconnect a battery from charging using grid power due to peak power demands.
  • the soft- switch may permit the battery to discharge to provide power to replace grid power in a local installation.
  • it may also prevent discharge or limit discharge from a vehicle battery if the grid power would otherwise draw from a vehicle battery to satisfy grid power demands.
  • the soft-switch may allow more of the battery to be discharged to the grid than it would do if the vehicle battery needs to reserve more energy for use at a later point in time.
  • the disclosed charging models are configured in some embodiments to allow for energy consumption to begin and end at multiple different times without the need for direct human intervention so as to take advantage of demand and dynamic pricing.
  • the method further comprises, responsive to determining that power should start to be provided to charge the vehicle to achieve the predicted target charged battery state by the determined future time, causing an alert to be provided to a user to connect the vehicle to a power supply if the vehicle is not already connected (S210).
  • at least one charging constraint is a charge or not condition based on an objective to minimize long term energy costs.
  • at least one charging constraint is a cost of charging constraint and/or an available power constraint. For example, if power cuts occur when demand exceeds a limit, this may be taken into account in the form of a charging constraint.
  • the set of power load variables of the external power source includes a power load variable of an external power supply to the charging power source, wherein the external power supply is connected to one or more other power-consuming devices consuming power in a charging period.
  • the charging constraint comprises a maximum available power constraint for the external power supply to provide power to the charging power supply.
  • the power-consuming devices include one or more devices which are not battery operated.
  • the charging model is trained to determine, for each of a plurality of charging periods, electricity cost and a probable risk of a battery state of energy not meeting a target charged battery energy state at the end of that charging period.
  • the method further comprises varying the amount of charging power used by the charging power supply to charge the vehicle battery based on the current vehicle battery state of energy.
  • the method further comprises obtaining data generated by implementing the dynamic vehicle battery charging model at the charging power supply, using the gathered data as a training data set for training the dynamic battery vehicle charging model, wherein the gathered data comprises, for each charging period, data representing the start time, the end time, the initial battery charge state, the end battery charge set, the power consumed charging the battery and the cost of the power consumed charging the battery to the charging model, and re- training the trained dynamic battery vehicle charging model by inputting the gathered data into the trained model.
  • Another, fifth, aspect of the disclosed technology comprises a method of training a charging model using reinforcement learning, the charging model being configured for use in a method of the first aspect or any one of its embodiments disclosed herein, the trained model being configured to determine when to charge the vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods, the method comprising inputting training data to the charging policy model, and iteratively processing the input training data to provide iterative output representing a charging cost, wherein each iteration comprises comparing a charging cost with a target charging cost or with a charging cost from a previous iteration and maximising cumulative rewards for reducing the charging cost to achieve the target battery charge state.
  • each reward, r, of the maximised cumulative rewards can be represented for each charging period as either a charge power consumption per unit time multiplied by a duration of time over which charge power is consumed multiplied by the energy consumption cost per unit of power in the case where the end of the charging period is reached and the target battery energy state is reached; or, if the target battery energy state is not reached at the end of the charging period, a total energy consumption cost for the duration of time the battery was charged plus a penalty term comprising the energy deficiency at the end of the charging period multiplies by a penalty constant.
  • each reward r of the maximised cumulative rewards is determined using a reward function based on a charge power consumption per unit time, a duration of time over which charge power is consumed, a battery energy deficiency comprising a difference between a target state of battery energy and a current state of battery energy, an energy consumption cost, and wherein the reward function indicates if an end of a charging episode is reached or not, and includes a penalty term defining how a violation from a target state of energy is to be penalised.
  • Another, sixth, aspect relates to an apparatus comprising a computer-program product comprising a set of instructions stored in a memory and comprising one or more modules or circuitry which, when loaded from the memory for execution by one or more processors or processing circuitry of the apparatus cause the apparatus to implement a method according to the first and/or second aspects or any of their embodiments disclosed herein.
  • the apparatus is configured to be connected to a grid power supply and comprises a charging power supply outlet configured to be connected to an actuatable a soft- switch 104, wherein, when an electric vehicle 100 is connected to the apparatus, the apparatus is configured to actuate the soft-switch to control a supply of electrical power to at least start charging a battery of the connected electric vehicle based on the apparatus implementing a method according to the first aspect or any one of its embodiments disclosed herein to determine an action to charge a connected vehicle should be performed.
  • actuating the soft-switch 104 also controls the supply of electrical power to stop charging a battery of the connected electric vehicle when the target charged state of energy of the battery is reached.
  • the apparatus comprises a battery charger.
  • Another, seventh, aspect of the disclosed technology comprises a computer-readable storage medium comprising computer-program code which, when executed by one or more processors or processing circuitry of an apparatus, causes the apparatus to implement a method according to the first aspect.
  • Another, eighth aspect of the disclosed technology comprises a computer program carrier carrying a computer program comprising computer-program code, which, when loaded from the computer program carrier and executed by one or more processors or processing circuitry of an apparatus causes the apparatus to implement a method according to the first aspect, wherein the computer program carrier is one of an electronic signal, optical signal, radio signal or computer- readable storage medium.
  • Another, ninth, aspect of the disclosed technology comprises a computer program product comprising computer-code which when loaded from memory and executed by one or more processors of a control circuit of an apparatus causes the vehicle to implement a method according to any one of the above disclosed method aspects and/or one or more of their disclosed embodiments.
  • Another, tenth, aspect of the disclosed technology comprises computer code for causing an apparatus to perform the method of the first aspect when said computer code is loaded from memory and run on one or more processors or on processing circuitry of a an apparatus.
  • the disclosed aspects and embodiments may be combined with each other in any suitable manner which would be apparent to someone of ordinary skill in the art.
  • Figures 1A and 1B schematically illustrate example power cost models over a 24 hour period
  • Figure 1C schematically illustrates a power management system for charging an electric vehicle according to some embodiments of the disclosed technology
  • Figures 2A and 2B schematically illustrate example charging episodes according to some embodiments of the disclosed technology
  • Figure 3A schematically illustrates a charging model according to some embodiments of the disclosed technology
  • Figure 3B schematically illustrates a charging policy based on a charging model to some embodiments of the disclosed technology
  • Figure 4A schematically illustrates an example of a method of power management for charging an electric vehicle according to some embodiments of the disclosed technology
  • Figure 4B schematically illustrates an example method of training a charging model using reinforcement learning according to some embodiments of the disclosed technology
  • Figure 5 illustrates schematically an example of an apparatus configured to perform a method of power management for charging an electric vehicle according to some embodiments
  • Electric vehicles require battery charging and usually to charge their batteries they are periodically connected to a mains electricity supply, often referred to as “the grid” which is provided using a power distribution network from an electrical power station which may, for example, comprise a hydro-electric, wind, nuclear or other type of electrical energy source.
  • the grid which is provided using a power distribution network from an electrical power station which may, for example, comprise a hydro-electric, wind, nuclear or other type of electrical energy source.
  • the demand for electrical energy from a mains electricity supply has well-known peaks and lows, and electrical energy providers often try to incentivise the use of electrical energy at off-peak times by offering favourable tariffs.
  • Electric vehicle batteries may be configured to store several 10s or more of kWhs of electrical energy. Typically battery charging is performed over long time-periods last several hours and in order to minimize the operating cost of the vehicle, it is advantageous if the cost of any electric energy bought from the grid is as cheap as possible.
  • the present disclosure relates to an intelligent system for charging one or more electric vehicles.
  • One solution seeks to use a charging policy for each vehicle based on historic use of an electric vehicle providing an indication of when at a future point in time a vehicle is likely to be required for use, and which may also determine, based on historic use, a minimum level of a state of energy, SoE, of a vehicle battery which would be required until the next point in time when battery charging is likely to take place.
  • SoE state of energy
  • a plurality of electric vehicles may be of the same or different types, and some may comprise heavy vehicles.
  • Examples of electric vehicles which may require more intelligent charging of their on-board battery systems include a plug-in electric vehicles as well as vehicles with a full electric powertrain, in other words, vehicles whose propulsion systems are solely powered using on-board battery systems.
  • Such vehicles may be convention vehicles driven by a human operator, with or without additional assistance, or semi- or fully autonomous vehicles.
  • Electric vehicles in a fleet may also include remote-operated electric vehicles in some embodiments of the disclosed technology.
  • FIGS. 1A and 1B schematically illustrate example power pricing models over a 24 hour period for charging a battery 101 of an electric vehicle 100 using a mains power accessed via a battery charger 104. As shown, the cost of charging the battery 101 of the electric vehicle varies over a 24 hour period.
  • Electric vehicles particularly when the vehicles are heavy-duty electric vehicles which may have larger battery capacity and require higher voltages to charge their batteries than smaller vehicles, may take several hours to fully charge depending on the battery power capacity and the rate at which the battery charger can recharge the battery.
  • charging the electric vehicle battery 101 begins at time T0 and the battery 101 finishes charging at time T end .
  • the battery will be charged over a period of time that crosses a plurality of charging tariffs as shown in Figure 1A where a examples of 1kWh pricing is between prices or costs #1, cost #2, and a final cost #3. These costs reflect in part the supply demand on the power distribution network which is illustrated schematically in Figure 1B.
  • Figure 1A shows an example plot of an electrical energy price model where the price of electricity is shown a function of time. In the example shown, it is more expensive to use electricity at evenings when cost #3 applies and demand is high (as shown in Figure 1B) than in the early morning when cost #1 applies.
  • Figures 1A and 1B accordingly show an example of a pricing model where a customer gets “punished” for using power during peak periods of demand. The power during a period is tracked and the electricity price is related to the peak power. This incentives householders and businesses for example to use electricity during off-peak rates and to avoid during peak-rates charging a vehicle battery. Moreover, it is also advantageous to avoid battery charging when there are other demands on the power being supplied, for example, made by machinery or household appliances.
  • FIG. 1C schematically illustrates a power management system for charging an electric battery 1010 of an electric vehicle 100 according to some embodiments of the disclosed technology which seeks to control if a vehicle should be charged or not in an adaptive and optimal manner.
  • the electric vehicle 100 for example, is a heavy-duty electric vehicle, whose battery 101 may be connected to a battery charger 106 via a soft-switch 104 which may be controlled either by the battery charger 106 or remotely by a server 102 configured to control operation of the battery charger and/or the soft-switch.
  • the soft-switch may comprise a relay which enables a battery of the electric vehicle 100 to be connected and/or disconnected from the electrical power supply provided by the battery charger.
  • the soft-switch is controlled using switch control software which sets the relay state of the switch according to one or more inputs the switch control software receives.
  • the relay of the soft-switch may according be fairly simple and not require any sophisticated power electronics.
  • this results in a soft-switch which may require little maintenance.
  • the disclosed technology uses a self-learning battery charging and power management system that seeks to manage power demands and/or grid electricity costs when charging a battery of an electric vehicle to find the optimal balance between energy cost and the risk of not ending up in a satisfying battery state of energy in a given charging episode.
  • the self-learning battery charging and power management system is implemented so that there is a minimum amount of user input and/or user awareness of the decision process the management system performs knowledge and so that during a charging episode, is a trade of between the electricity cost and the risk of not ending up in a satisfying battery state of energy.
  • FIGs 2A and 2B schematically illustrate example charging episodes according to some embodiments of the disclosed technology where power is illustrated as a solid line and the battery state of energy SoE is shown as a dashed line.
  • the target SoE of the battery is 100%.
  • the charging policy implemented was too risk adverse, in other words, it was too careful and overly prioritized short term energy cost savings and as a result, was exposed to higher energy costs later in order to reach the target SoE by the target future point in time, which is 8:00am in the example illustrated.
  • a better balance between electricity cost and the risk of not ending up in a satisfying battery state of energy is achieved.
  • the battery state of energy in other words, the battery SoE is a normalised number which is 100% for a fully charged battery.
  • the SoEerror is the target SoE minus the actual SoE.
  • action is a Boolean decision taken by a charging policy which is true if charging is recommended and t norm is normalized time, the spend charging time divided by the time in the day of the charging.
  • the charging apparatus 106 uses a non-rule-based charge policy, for example a charge policy configured to use one or more pattern recognition techniques or machine learning algorithms to determine an action to perform, for example, an action which determines when charging should start and/or end in order to charge a battery of an electric vehicle to a target SoE at a future point in time.
  • the policy maps a set of variables, for example SoE ⁇ and t ⁇ , to an optimal action comprising whether the vehicle charger shall receive charging power or not.
  • the charging power is referred to as grid connected power or grid power in embodiments where electrical energy is received from a power distribution network or grid mains power supply.
  • the charging power may be partly or wholly indirectly grid powered if a storage battery configured to be grid powered for back-up purposes is used.
  • a battery may also be connected to a local power generation source such as photo-voltaic array of solar panels or a wind turbine.
  • Figure 3A illustrates schematically an example embodiment of such a charge policy.
  • input to the charging model is processed by the charging model and the model generates output in the form of an action to charge a battery or to not charge the battery in a way that optimizes a reward function, r, and which updates a state transition function, s ⁇ .
  • the reward function uses cumulative rewards, for example, as follows:
  • the terms used in the above reward function, r are defined in the table below as follows:
  • the transition function of the charging model which expresses how a state s ⁇ is to be updated is defined by: 1) ⁇ ⁇ ⁇ ( ⁇ , ⁇ )
  • the specific problem of charging it includes, for example, the relation 2) ⁇ ⁇ ⁇ + ⁇ ⁇ ⁇ ⁇
  • the value of an action in a particular state can be used. Reinforcement learning involves an agent, a set of states, S, and a set of actions A per state.
  • Some embodiments of the charging model may use a reinforcement learning algorithm where the agent seeks to maximize its total cumulative reward by adding the maximum potential reward attainable from future states to its reward for achieving its current state, where the potential reward is a weighted sum of expected values for the rewards in all future steps starting from the current step. This influences the current action by the potential future reward.
  • a charging model could learn that, even if initially for a given set of input variables where the reinforcement learning algorithm output an action to charge the battery starting from a time tx which resulted in a negative reward, for example, as the target SoE is not reached at the end of a charging period, that repeatedly charging the battery starting at that time tx results in in the long term that starting at time tx is better than at other start times, as the over-all reward is better with the start time tx.
  • a model-free reinforcement learning algorithm may be used such as, for example, a Q-learning reinforcement learning algorithm is used which does not require a model of an environment and which can handle stochastic transitions and rewards without requiring adaptations.
  • the “Q” being learnt is the reward function computed by the learning algorithm for an action in a given state.
  • a Q-learning reinforcement learning algorithm seeks to find an optimal policy starting from a current state S for any finite Markov decision process which maximizes the expected value of the total reward over any and all successive steps, starting from a current state. In other words, by repeating possible actions for which a reward is provided a sufficient number of times, actions which result in optimum rewards can be learnt.
  • embodiments of the charging model which use reinforcement learning do not have to iterate for every time step as learning can be done offline and optimal solutions stored in memory, for example, in a look-up table such as that shown in Figure or such as in a neural network, from which optimal control actions can be retrieved almost instantaneously.
  • Such offline learning may provide a technical effect as by needing to iterating every time-step and learning off line, computational resources can be managed in a more energy efficient way.
  • the Q- learning algorithm has a function that calculates the quality of a state-action combination as follows: Q: S ⁇ A ⁇ R Q is initialised to a value before learning begins and then each time t the agent selects an action at ⁇ A, a reward, rt, is calculated and the agent enters a new state st+1, which may depend on both the previous state st and the selected action, and Q is updated.
  • Some Q-learning algorithms may use a Bellman equation to provide value iteration updates using the weighted average of the current Q value and new information. An episode of the algorithm ends when st+1 reaches a final state. The temporal difference is the time-step.
  • Some embodiments of the invention use a simple tabular Q-learning algorithm which can be expressed in pseudo-code by: [000108] The “Q” which is being learnt, in other words, the reward function computed by the learning algorithm for an action in a given state, provides an indication of how good or bad it is to take a specific action.
  • Q expresses the long-term consequence of an action, the cumulative reward, while r is the short term reward.
  • a simple look-up table can be used as memory for holding the Q values learnt for each new state s’ after taking an action a for the tabular Q-learning algorithm.
  • the model algorithm is trained off-line using recorded data.
  • a buffer may be provided with recorded price data, rpdb.
  • rpdb For every charging episode, almost once per day for daily commuters using an eV, a price data trajectory is added and stored in rpdb.
  • a naive policy may be used in some embodiments, for example, a policy which always recommends charging.
  • the initial training may start and a draft policy may be recommended based on the initial training. Training can be repeated so that a draft policy is recommended, maybe after 20-30 charge episodes.
  • FIGS. 4A and 4B show example embodiments of how the server 102 or the battery charger apparatus 106 may implement a method which uses a charging policy model such as one of the above examples which use a Q-learning algorithm to determine when to supply charging power so that a battery is charged to a target SoE by a target future point in time, for example, by the server 102 or battery charger 106 activating the soft-switch 104 shown in Figure 1C to charge a battery of a vehicle 100.
  • a charging policy model such as one of the above examples which use a Q-learning algorithm to determine when to supply charging power so that a battery is charged to a target SoE by a target future point in time, for example, by the server 102 or battery charger 106 activating the soft-switch 104 shown in Figure 1C to charge a battery of a vehicle 100.
  • the vehicle 100 may be any type of electric vehicle with a battery that can be recharged from an external battery charger.
  • Examples of vehicle 100 include a heavy-duty vehicle or accessory having an electric battery, which may be a plug-in hybrid battery or a battery for an electric vehicle which does is the main or sole source of power for propelling the vehicle.
  • Figure 4A shows an embodiment of a computer-implemented method 200 for charging an electric vehicle at a charging power source
  • the method comprises detecting a current state of energy of a battery of a vehicle (S202) determining a target charged state of energy of the battery at a future point in time (S204) and determining a point in time when power should be provided to charge the vehicle from the charging power source based on the detected current state of energy of the battery and the determined target charged state of energy of the battery (S206) so that the future point in time is a predicted end of a charging time period, wherein the future point in time is determined using a dynamic vehicle battery charging model configured to determine the target charged battery energy state based on a predicted user demand for power from the charged vehicle at the end of the charging period and at least one charging constraint for the charging power source.
  • Method 200 is performed by the battery charger 106 and the detected vehicle state is a measured of sensed battery vehicle state which may be obtained via the soft-switch 104 in some embodiments.
  • the future point in time may be learnt by the charging algorithm of the charging model or input by a user of the vehicle to the battery charger in some embodiments. In other words, given a known future point in time when the vehicle is to be driven, the goal of the charging model is to minimise the financial cost of charging the battery so that it is ready for use with a minimum desired state of energy of the vehicle battery at that point in time.
  • the dynamic vehicle battery charging model may use a reinforcement learning algorithm with cumulative rewards such as one of the above mentioned examples which may use tabular Q-learning or neural-network Q-learning to provide a cumulative reward for actions which charge the battery SoE to the target SoE by a particular future point in time, which may be user specified or which may learnt from historic charging cycles.
  • the charging model receives a set of power load variables for the charging power source and processes these to determine an action to charge the battery or a non-charging action to not charge the battery by rewarding for long-term and short-term consequences of the charging or non-charging actions.
  • the charging system By learning usage patterns for the electric vehicle it allows the charging system to be a self-learning system in some embodiments that minimizes grid electricity expenses for battery charging without risking the vehicle not having a sufficient amount of charge when it is needed to be used.
  • the method 200 further comprises, if the model determines an action that battery is to be charged based on the determined current state of energy of the battery and the charging model, configuring the external charging power supply to automatically provide power to charge the vehicle by actuating a soft switch or relay S208.
  • the method may further comprise, responsive to determining that power should be provided to charge the vehicle to reach the predicted target charged battery state by the required time of day, causing an alert to be provided to a user to connect the vehicle to a power supply if the vehicle is not already connected S210.
  • an alert to be provided to a user to connect the vehicle to a power supply if the vehicle is not already connected S210.
  • the charging model used by method 200 may in some embodiments be trained and/or retrained by using a set of power load variables for the battery charger, a term which may also refer to any other suitable form of external power source 106 which takes grid power directly or indirectly and uses it to charge the vehicle.
  • the set of variables include a power load variable on the external power supply for each of one or more other power- consuming devices such as household appliances or machinery consuming power from the electrical installation connected receiving grid power as the battery charger 106in a charging period.
  • the maximum grid power drawn via the electrical installation can be managed and any limits not exceeded. This may help prevent overloading and/or one or more appliances, including the battery charger to failing to operate normally.
  • the power- consuming devices may be battery-operated and connected to battery chargers connected to grid- power in some embodiments.
  • One or more power-consuming devices may use the same battery charger 106 used to charge the battery of the vehicle 100 in some embodiments. However, the power-consuming devices may include one or more devices which are not battery operated and directly connected to the grid in some embodiments.
  • the method 200 uses charging model which is trained to determine, for each of a plurality of charging periods, electricity cost and a probable risk of a battery state of energy not meeting a target charged battery energy state at the end of that charging period.
  • the amount of charging power used by the charging power supply to charge the vehicle battery based on the current vehicle battery state of energy may be varied in some embodiments, for example, based on the price cost of the electricity consumed and/or responsive to other demands on the electrical installation to which the battery charger is attached and/or to other demands on grid power.
  • the charging model is configured to modify the reward function to disincentivise the use of power during peak periods in some embodiments.
  • the charging model vary the amount of charging power to minimise the electricity cost for charging the battery over one or more charging periods.
  • a charging model which uses Q-based learning algorithm where Q represents cumulative rewards comprising short and long term rewards based on the consequences of charging the battery meeting the cost constraints or not may be used in some embodiments.
  • the charging model which is implemented using the method 200 shown in Figure 4A is configured to use a Q-based learning algorithm configured to determine whether to charge the battery or not using cumulative rewards for reducing the charging cost whilst still achieving a target SoE by a desired future point in time.
  • the battery charger may cause the battery to not be charged when a user wants to use the vehicle.
  • Training may use historic user behaviour data obtained via the charging station 106 or another charging station and/or historic vehicle use data obtained from the vehicle or a remote data repository in some embodiments.
  • method 200 implements a dynamic charging model configured to be retrained from time to time using data gathered from implementing the model at a charging power supply and sends data for each charging period representing the start time, the end time, the initial battery charge state, the end battery charge set, the power consumed charging the battery and the cost of the power consumed charging the battery to the charging model S212.
  • Figure 4B illustrates an example of how the charging model may be trained or retrained according to some embodiments of the disclosed technology in which the charging model is trained using a method 300 which implements a reinforcement learning algorithm.
  • the training data may take the form of electrical energy pricing and demand data over a given time period such as 24 hours, sampled over a longer time period, for example, weeks or months or longer.
  • the data shown in Figures 1A and 1B shows schematically examples of data which may be used to train the charging model.
  • the charging model which is configured using method 300 may be used as a trained charging model in an embodiment of method 200.
  • the trained charging model is configured to determine when to charge the vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods by inputting training data to the charging policy model S302 and iteratively processing the input training data to provide iterative output representing a charging cost S304.
  • Each iteration performed comprises: comparing a charging cost with a target charging cost or with a charging cost from a previous iteration S306 and maximising cumulative rewards for reducing the charging cost S308 to achieve the target battery charge state.
  • Each reward, r, of the maximised cumulative rewards can be represented for each charging period as either a charge power consumption per unit time multiplied by a duration of time over which charge power is consumed multiplied by the energy consumption cost per unit of power in the case where the end of the charging period is reached and the target battery energy state is reached or, if the target battery energy state is not reached at the end of the charging period, by a total energy consumption cost for the duration of time the battery was charged plus a penalty term comprising the energy deficiency at the end of the charging period multiplies by a penalty constant.
  • method 300 may comprise a method 300a which initially trains the self-learning charging model off-line using training data comprising price data trajectories for a plurality of historic charging episodes/events.
  • the method 300a may also, after each battery charging episode, gather as new training input data the input data to the charging model associated with the charging episodes and the charging model data associated with each charging episode ending which is output in S310, and, after a number of battery charging episodes, updating the training data S301 and retraining the charging model using method 300b.
  • charging model may be implementing using a computer-program product in some embodiments by an apparatus comprising the server 102 and/or battery charger 106.
  • the computer-program product may comprise a set of instructions stored in a memory and comprising one or more modules or circuitry which, when loaded from the memory for execution by one or more processors or processing circuitry cause a charging model trained using an embodiment of one or more of the methods 300, for example as shown in Figure 4B, and then once trained, implemented in an embodiment of method 200 as shown in Figure 4A.
  • the trained charging model includes a set of power load variables for a charging power supply which comprise the following variables: a variable representing a current time of day, a variable representing a charging tariff for power drawn based on the current time of day, a variable representing a target charged battery energy state, and a variable representing a current energy state of the battery.
  • executing the instructions causes the charging model to map a current battery state of energy, a target battery energy state, and the duration of battery charging divided by the time of day spent charging to determine a charging cost to reach the target battery energy state.
  • the charging model may comprise a self-learning model trained using a reinforcement machine learning to determine when to charge an electric vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods.
  • the reinforcement learning model is configured to maximise cumulative rewards for reducing the charging cost, and may comprise a Q-learning algorithm such as one of the example Q-learning algorithms disclosed herein above.
  • Such a reinforcement learning model is configured to maximise cumulative rewards for reducing the charging time.
  • the charging model is a dynamic charging model which is configured to be retrained from time to time using data gathered from implementing the model at a charging power supply, such as Figures 4A and 4B show collectively.
  • the battery charger 106 may be referred to herein as a battery charger or charging apparatus may be used to implement the charging model in some embodiments.
  • server 102 is used to implement the charging model.
  • Each of the battery charger 106 and/or the server, also referred to herein as server apparatus, 102 comprise suitable memory, one or more processors or processing circuitry, and computer code stored in the memory, so that by executing the computer code loaded from the memory the battery charger and/or server apparatus may be respectively configured to implement a method of charging a battery 101 using a trained learning charging model which uses a cumulative reward system according to the disclosed embodiments.
  • the computer code comprises a set of instructions stored in the memory, the set of instructions comprising one or more modules or circuitry such as Figure 5 shows schematically for the battery server 106 and Figure 6 shows schematically for the server 102.
  • Figures 6 and 7 it is assumed that method 200 is performed by the battery charger 106 and the training method(s) 300(a,b) are performed by the server 102.
  • the training method 300a it may be possible for the training method 300a to be initially performed by server 102, which then sends an initially trained charging model to the battery charger 106 such as Figure 5, described in more detail below shows, but later for the charging model to self-learn in some embodiments.
  • the initial pre-training may not take place, and the battery charger 106 only learns what is the best cause of action to follow. This may result in some inconvenience to users until the charger learns when use of the electric vehicle requiring a certain minimum level of battery charge is likely, as the user may have to adapt their journey to allow for recharging and/or use more conventional fuels in the case where hybrid vehicles are used.
  • the battery charger 106 may from time to time, receive retrained charging models from server 102 and/or may receive variables to update its charging model.
  • the apparatus comprises at least one or more processors or processing circuitry 402, memory 404 which holds computer code configured when executed to implement a method of charging a battery as disclosed herein.
  • the method of charging the battery be implemented by executing code in the form, for example, of modules M202, M204, M206, M208, M210, and M212 where each module causes a corresponding method feature S202, S204, S206, S208, S210, S212 as shown in Figure 4A and described hereinabove to be implemented in some embodiments.
  • module S202 may be used to detect a current state of energy of a battery of a vehicle for example such as a battery of a vehicle which may be recharged using a battery charging to implement an embodiment of method 200 as disclosed herein. Accordingly, in some embodiments collectively modules S202- S212 cause the method 200 comprising method features S202, S204, S206, S208, S210, S212 as shown in Figure 4A and described herein above to be implemented.
  • the battery charger 106 is a communications enabled battery charger including a suitable communications module and receiver/transmitter circuitry 406 and may include an antenna to support wireless communications using one or more communications protocols with at least server 106 and soft-switch 104.
  • cellular or Wi-Fi communications protocols may be used to communicate with server 102 via a suitable cellular network or Wi-Fi network access point in some embodiments whereas Wi-Fi or BluetoothTM communications protocols may be used to communicate with soft-switch 104.
  • executing the charging model algorithm to implement the method 200 may result in an action to charge or stop charging the electric vehicle at a particular point in time, and this decision may be actuated by a soft-switch controller 408 is some embodiments causing the relay of the soft- switch to be actuated.
  • the controller 408 may cause a control signal to be sent to the soft-switch using either as a wireless signal or over a wired communications link.
  • the soft-switch 104 may be provided as a component part within the battery charger 106 however, in some embodiments, the soft-switch 104 may be retrofitted and configured as a separate component.
  • Figure 6 shows an example embodiment of server apparatus 102 comprising one or more processor(s) or processing circuitry 502, memory 504 which holds computer code 510 which, when executed, causes the apparatus 102 to implement one or more training methods 300, 300a, 300b for configuring a charging model for a battery charger according to any of the disclosed embodiments.
  • the charging model is configured to be trained using computer code provided in the form of one or more modules M301,M302,M304,M306, M308, and M310, where each module comprise code configured to implement the corresponding S301, S302,S304, S306, S308, and S310 features of the training method shown in Figure 4B and described hereinabove.
  • Figure 5 shows schematically an example data flow when training a reinforcement learning charging model according to some embodiments of the disclosed technology.
  • the charging model Q-learning algorithm is initially trained by the server 102.
  • This initial training may be advantageous as it helps limit or avoid any inconvenience to a user whose vehicle battery is less likely to reach a target SoE in the early stages of the battery charger’s self-learning unless the self-learning model is pre-trained to some extent.
  • the server 102 sends the trained model to the battery charger 100 which provides charging power supply to vehicle 100 via a soft-switch 104.
  • the server 102 is able configure and reconfigure the charging model which is hosted on the battery charger 106 configured to provide a charging power supply to a connected vehicle 100 whenever the charging model outputs an action to charge the vehicle.
  • the battery charger 106 sends an actuation control signal to actuate the soft-switch to allow the vehicle (assuming it is connected) to be charged.
  • actuation control signal to actuate the soft-switch to allow the vehicle (assuming it is connected) to be charged.
  • a check is performed to confirm if a vehicle is connected or not and an alert may be generated to flag to a user associated with the battery charger and/or vehicle to connect the vehicle for charging.
  • the soft switch 104 may be configured to detect when a vehicle connects for subsequent charging and it may be configured to obtain the current state of battery energy.
  • these steps may be performed by the battery charger via the soft- switch and the soft-switch only actuated to provide a power supply when actuated by the battery charger’s soft-switch controller 408. which is actuated by the charging power supply according to the model.
  • the soft-switch 104 sends connected vehicle battery information to the battery charger 106 which inputs this into the trained charging model.
  • the charging model then processes the input and outputs an action to charge or not charge the vehicle, or for example, when to start charging the connected vehicle.
  • the battery charger Once conditions are met to start charging, the battery charger generates a control signal to actuate the soft-switch 104 and once the soft-switch receives the control signal it actuates and allows the vehicle 100 to charge.
  • the vehicle may be an autonomous electric vehicle with an ADS configured to make tactical decisions for a control system, for example, it may determine when to dock at the battery charger for recharging.
  • the electric vehicle may be a heavy, also known as a heavy- duty, electric vehicle.
  • a heavy-duty electric vehicle may comprise a wide range of different physical devices, such as combustion engines, electric machines, friction brakes, regenerative brakes, shock absorbers, air bellows, and power steering pumps. These physical devices are commonly known as Motion Support Devices (MSD).
  • the MSDs may be individually controllable, for instance such that friction brakes may be applied at one wheel, i.e., a negative torque, while another wheel on the vehicle, perhaps even on the same wheel axle, is simultaneously used to generate a positive torque by means of an electric machine.
  • the operation, particularly the autonomous operation, of a heavy-duty electric vehicle is accordingly more complex than the operation of a more light-weight vehicle such as a car and the power requirements more complex.
  • a heavy-duty vehicle may be electrically connected to a vehicle accessory which may also be charged via the battery charger 106.
  • the charging model may also monitor separately and/or collectively the state of charge of a connected vehicle accessory to the vehicle connected to the soft-switch and may also charge a battery of the connected vehicle accessory using a separate charging model for that vehicle accessory or using the same charging model as for the vehicle the accessory is electrically coupled to.
  • Some, if not all, of the above embodiments may be implemented using computer program code which may be provided as software or hardcoded, for example, as a computer program product configured to be used by a device mounted on or integrated in the battery charger 106 or server 102.
  • the methods 200, 300 described above may be at least partly implemented through one or more processors, such as, the processors or processing circuitry 402, 502 shown in Figures 5 and 6 together with computer program code 410, 510 for performing the functions and actions of the embodiments herein.
  • the program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code or code means for performing the embodiments herein when being loaded into the processing circuitry in a suitable controller or control unit of the apparatus 106, 104.
  • the data carrier, or computer readable medium may be one of an electronic signal, optical signal, radio signal or computer-readable storage medium.
  • processing circuitry and the memory or computer readable storage unit described above may refer to a combination of analog and digital circuits, and/or one or more processors configured with software and/or firmware, e.g. stored in a memory, that when executed by the one or more processors such as the processing circuitry perform as described above.
  • processors as well as the other digital hardware, may be included in a single application-specific integrated circuit (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a-chip (SoC).
  • ASIC application-specific integrated circuit
  • SoC system-on-a-chip
  • the controllers 408, 508 may also comprise or be capable of controlling how signals are sent wirelessly via antenna 70 in order for the vehicle 12 to communicate via one or more communications channels with remote entities, for example, a site back office.
  • the communication channels may be point-to-point, or networks, for example, over cellular or satellite networks which support wireless communications.
  • the wireless communications may conform to one or more public or proprietary communications standards, protocols and/or technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol for email (e.g., Internet message access protocol (IMAP) and/or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), and/or Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS)), or any other suitable communication protocol, including communication protocols
  • the operating systems of the apparatus 106, 108 and of the soft-switch 104 may further various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and for facilitating communication between various hardware and software components.
  • Figure 8 shows an example embodiment of a further aspect of the disclosed technology which relates to a charging model which seeks to control, in an adaptive and optimal manner, which electric vehicles of a plurality of electric vehicles shall have their batteries 101a,b,c charged or not charged in any given time period.
  • a power source 118 for example, a gas or coal or nuclear power plant, or hydroelectric or wind-powered source, is configured to supply electricity which is distributed via a series of transmission sub-stations 116 (only one is shown in Figure 8 for clarity) to a distribution sub-station 114 configured to provide power to one or more sub-areas 120 of the grid network 110.
  • a power source 118 for example, a gas or coal or nuclear power plant, or hydroelectric or wind-powered source
  • an example of a local power distribution area 120 comprises a server 102, and a plurality of battery chargers 106a,b,c, each controllable via a software switch 104a,b,c for recharging a battery system 101a,b,c, of an attached electric vehicle such as an electric heavy vehicle 100a,b,c.
  • the plurality of electric vehicles 100a, 100b, 100c may form a fleet of electric heavy vehicles and/or electric heavy vehicle accessories in some embodiments of the disclosed technology which are configured to communicate with the server 102, for example, a back-office server for the fleet of vehicles.
  • an electric heavy vehicle may have one or more connected accessories such as a trailers or container attached and electrically coupled to the electric heavy vehicle. Such accessories may be configured to share the vehicle’s battery energy resource and/or use their own separate batteries
  • An electric vehicle with one or more such electrically connected accessories is connected to battery charger 106 via soft-switch 104, it may accordingly charge both its own on-board battery 101 as well as the battery of a connected vehicle accessory in some embodiments.
  • a charging model may be used to generate a charging policy for determining when each vehicle’s battery (and/or the battery of any connected accessories in some embodiments) should be charged in a given time-period.
  • the charging model may treat a battery of a connected vehicle accessory as part of a battery system including the vehicle battery 101.
  • the accessory may be regarded as just another electric vehicle and the accessory’s battery may be regarded as a battery of another electric vehicle in the fleet of vehicles by the charging model.
  • each electric vehicle 100a,b,c may be autonomous or semi- autonomous electric vehicles or remotely operated electric vehicles in some embodiments. Such electric vehicles 100a,b,c, will also need to be charged from time to time to ensure their battery systems have sufficient energy for them to go about their allocated tasks at allocated times.
  • Each electric vehicle may each be connectable via a soft-switch or similar relay apparatus 104a, 104b, 104c to a corresponding individual battery charger 106a, 106b, 106c.
  • Each of one or more or all of the vehicles 100a-100c may be concurrently connected to a respective one of the plurality of battery chargers 106a, 106b, 106c via a respective one of the soft-switches 104a- 104c, however, being connected does not automatically actuate the soft-switch 104a-104c so that electrical power is provided to charge the vehicle batteries by the battery chargers 106a-106c.
  • each soft-switch or relay 104a, 104b, 104c functions in a similar away to soft-switch 104 illustrated in Figure 1C described above in that it can be actuated using an electrical control signal to allow a connected vehicle battery 101a,b,c, to be charged from grid power by each respective vehicle 100a, 100b, 100c.
  • Each soft-switch 104a,b,c can also be actuated using an electrical control signal to disconnect a battery 101a,b,c, from the grid power distribution network 110.
  • a software agent executing on a remote server 102 may be configured to commanding the soft-switches or similar relays 104a,104b,104c based on data communicated from the battery charging apparatus 106a, 106b, 106c.
  • Server 102 may communicate with the soft-switches or relays 104a,b,c directly and/or via the battery charging apparatus, using a suitable wired or wireless communications protocol so as to control actuation of the soft-switches or relays depending on the charging actions determined by the charging model executing on server 102.
  • the dashed line between battery chargers 106b, 106c and transformer 112a represents a power transmitting cable 130 connected between a power distribution grid network 110 and a power distribution sub area 120 such as, for example, an industrial customer power distribution area 120 configured to supply power to server 102 and battery chargers 106a,b,c.
  • a network electricity sub-station 114 in the power distribution network 110 will typically supply power to this type of electrical power distribution sub-area 120 to cover power demands of up to around 2000 kW/h, or 2 Megawatts/h per sub-area of the grid.
  • each electricity sub- station may be configured to supply only one distribution sub-area, but in others, it is possible for other sub-areas to be supplied with power.
  • the electrical power used to charge a battery 101a,b,c, of an electric heavy vehicle 100a,b,c, such as the trucks shown schematically in Figure 8 may be of the order of 50kW/hour to 500kW/hour. Based on a fleet comprising, for example, twenty or so such electric heavy vehicles, each electric heavy vehicle requiring 20 kW/hour of power to charge its battery system 101, results in a potential power demand of typically ⁇ 1000 kW/hour for however many hours it takes to charge the battery systems of all of the electric heavy vehicles in the fleet of vehicles.
  • FIG. 8 shows three electric vehicles 100a,b,c
  • any number of electric vehicles and/or vehicle accessories having batteries which may need to be recharged from time to time may operate and need to use battery chargers 106a,b,c, in a sub-area 120 at any given point in time.
  • the disclosed embodiments should not be limited to just three vehicles 100a,b, c as shown in the example embodiments of Figure 8.
  • the power distribution network 110 As the number of vehicles in a fleet rises, so too will the need to manage how much power is drawn at any given time from the power distribution network 110.
  • the self-learning system disclosed herein above with reference to Figures 1 to 7 and the accompanying description hereinabove sought to use a reinforcement learning based charging model to implement a charging policy to manage the timing of energy consumption for recharging a single vehicle so as to manage grid electricity expenses and ideally to avoid recharging during periods of peak demand, which are more costly accordingly, for a single vehicle being charged.
  • a reinforcement learning based charging model to implement a charging policy to manage the timing of energy consumption for recharging a single vehicle so as to manage grid electricity expenses and ideally to avoid recharging during periods of peak demand, which are more costly accordingly, for a single vehicle being charged.
  • such a model requires certain adaptation when extending to how to manage charging multiple vehicles in a way that manages power demands and/or electricity costs.
  • the disclosed technology uses the techniques 200 disclosed hereinabove for managing recharging a battery of a single electric vehicle using a charging model which may be trained for example, using a method such as method 300 disclosed herein to determine if any of the plurality of vehicles 100a,b,c should be first connected to the power distribution network 120, and then if an electric vehicle is determined by the charging model implemented using method 200 to be allowed to be connected to receive charging power from the power distribution sub-area network 120, and then it identifies which vehicles should be connected to which individual power stations 106a,b,c, so as to receive charging power during one or more time-periods.
  • a sub-set of the electric vehicles which form a fleet of electric vehicles can be identified for connection to individual battery chargers 106a,b,c by at least determining or using a known maximum power limit for the amount of power that should be provided to the fleet of electric vehicles by the set of battery chargers 106a, 106b, 106c and then maintaining the amount of power that is transmitted to recharge the batteries of the fleet of vehicles below the maximum power limit.
  • the disclosed technology seeks to minimize a standard deviation of the battery state of energy levels so these all remain within acceptable levels. In other words, ideally no vehicles should run out of battery energy whilst another vehicle has its battery recharged to full capacity based on the charging model.
  • the charging model seeks to minimize the total deviation from a target SoE of vehicle batteries across the fleet of vehicles. In other words, ideally no vehicle should have their battery capacity fall below a certain percentage of their target SoE in some embodiments. In some embodiments, the certain percentage target SoE may be replaced or accompanied by a lower threshold for battery SoE.
  • n reflects the number of vehicles in the fleet
  • Pch i refers to the power consumed charging the ith vehicle of the fleet to the target SoE for that vehicle
  • Pch max fleet refers to the maximum amount of power that the fleet can draw
  • SoE is a battery state of energy, a normalized number.100% for a fully charged battery
  • SoE error is the target SoE minus the actual SoE
  • nb(x) is a function zero for a negative argument, else the argument itself.
  • Nb(x) acts accordingly as a negative values blocker.
  • Action refers to a Boolean decision taken by a charge policy, which is true if charging of any vehicle is recommended
  • W is a weighting hyper parameter, which may be initially set to one.
  • u is the control variable, expressing the connection policy for whether a vehicle I of the plurality of n vehicles be connected to the grid power supply or not. It is a binary string. Every position in the string corresponds to a vehicle.
  • a binary value of one may mean charge in some embodiments.
  • the values of u may be based on the actions or control logic output by a charging model such as a charging model described herein above which is self-learning and which, once trained, may be used to implement a charging policy.
  • a charging model such as a charging model described herein above which is self-learning and which, once trained, may be used to implement a charging policy.
  • the decisions may be stored in a look-up table or in neural networks in some embodiments which allows the charging model to be implemented off-line and just the decisions retrieved later from memory when the variables input to the charging model match input conditions associated with a stored charging logic value.
  • t+ ⁇ t indicates the cost is to be minimised taking into account the consequences at some time ahead at the end of the charging period ⁇ t, of now (as in the current time-step) taking a specific control action, for example, if a specific u setting is applied in the current time-step of the charging model.
  • Figures 9A and 9B illustrate schematically consequences of letting two vehicles 100a, 100b be charged for ⁇ t minutes in the form of a graph of % battery SoE (y-axis) vs charging time period ⁇ t (x-axis).
  • a SoE of each of three electric vehicles 100a, 100b, 100c for example, the electric vehicles 100a, 100b, 100c shown in Figure 8 and described above for example is shown as a % of the full battery energy capacity.
  • the charging policy implemented at the start of any charging time period may affect the fleet performance in future at the end of the charging time period. This is illustrated schematically in Figures 9A and 9B which where each differently filled column shows a % SoE for each vehicle 100a,b,c.
  • the % SoE of the battery system 101a of vehicle 100a may be represented by the left-hand side column of the chart in Figures 9A and 9B with a horizontal line fill
  • the % SoE of the battery system of vehicle 100b may be represented by the middle column which has a dotted fill
  • the % SoE of vehicle 100c may be represented by the right hand side column with a plain/no fill.
  • Figures 9A and 9B show two different examples of how the % SoE of vehicles 100a and 100b may change from the start of a charging time period ⁇ t to the end of the charging time period based on the charging policy implemented for the fleet of vehicles.
  • the vehicle 100c is not charged during the period shown as it has a SoE above its target SoE.
  • the darker shaded areas in Figures 9A an 9B illustrate SoE target deviations.
  • the charging model appears to be providing optimal control over when each vehicle 100a,b is charged and to what SoE, because on the left hand side of the charge, at the end of the charging period, both the tad, the sum of SoE target deviations has decreased and the standard deviation, in other words, the spread, of the future actual SoE values, have decreased.
  • the charging model appears to be making good decisions for the fleet of vehicles.
  • Figure 10 A shows a flow chart for a method 600 of managing the charging of a plurality of vehicles, for example, n vehicles.
  • a method 600 of power management for charging a plurality of batteries of a fleet of electric vehicles comprises determining a set of variables for each vehicle in the fleet, the set of variables including a current battery state of energy variable and a target battery state of energy at the end of a charging period in S602, and determining in S604 if power should be provided to charge one or more vehicles in the fleet using at least one external power source using a self-learning charging model.
  • An example of a self-learning charging model which may be used to implement the disclosed technology is one which uses reinforcement learning such as an embodiment of the Q-learning charging model described herein above,.
  • Such a model may be configured to map the set of variables to an action to charge the vehicle battery or not to charge the vehicle, based on one or more charging constraints for the fleet of vehicles.
  • variables include a current time of day, a charging tariff for the current time of day, the target charged battery energy state and the current energy state of the battery.
  • the charging constraints may comprise one or more charging constraints of the method 200 described herein above.
  • a charging constraint for the fleet of vehicles may comprise a cost constraint for charging the fleet of vehicles.
  • a charging constraint for the fleet of vehicles comprises a power constraint representing a power limit for the amount of power that can be transmitted to the fleet of vehicles at any given point in time.
  • the charging model is configured to minimize a standard deviation of battery state of energy levels across the fleet of vehicles.
  • the charging policy is configured to minimize a total deviation from a target battery state of energy level across the fleet of vehicles.
  • the current battery state of energy is determined when that vehicle is connected to an external power supply and the method further comprises, if that vehicle is determined to receive power using the charging policy, configuring the external power supply to provide power to charge the vehicle.
  • the external power supply may be configured to provide power to charge a battery of an electric vehicle and/or an electrically connected vehicle accessory by actuating a soft switch or relay.
  • the method 600 may accordingly include configuring an external power supply such as a battery charger 106a,b, c to provide power to charge a vehicle battery system by actuating the soft switch or relay 104a, b,c in some embodiments.
  • the charging model policy may be configured to determine, for each of a plurality of charging periods, an electricity cost and a probable risk of a battery SoE not meeting a target charged battery SoE at the end of each charging period.
  • the charging model used in the above method 400 may be used to implement a charging policy such as the example the charging policy 500 shown schematically in the flow- chart of Figure 10B.
  • a method 700 of implementing a charging policy for power management when charging a plurality of batteries of a fleet of vehicles comprises in S702 determining if any vehicle is allowed to be connected to the grid, for example, based on inputting the average of present SoE levels of the vehicles of the fleet into a charging model.
  • the variables input to the charging model such as those indicated above as variables of method 400, result in an output with an action value of 1, then instead the method must determine which vehicles of vehicles 100a,b,c shall be individually able to receive charging power from their respective battery chargers 106a,b,c by actuating soft-switches 104a,b,c, in S706.
  • S506 may be implemented by determining a binary control string which controls which individual vehicles, say vehicle 100a, 100b should receive power by actuating the soft switches 104a, 104b of respective battery chargers 106a,106b in S708.
  • the action values 0, and 1 indicated in Figure 10B represents the charge logic determined from the output of the charging model, in other words, the output determined using a suitable reinforcement learning algorithm in some embodiments which may be trained using data such as that shown in Figures 1A and 1B obtained for the plurality of different electric vehicles forming the fleet which indicates if charging should happen at a given point of time to optimize the model and ideally achieve a target SoE for each vehicle in the fleet.
  • the charge or not logic presented by the action value may set charging power restrictions for all of the plurality of vehicles, in other words, for the full fleet.
  • This may allow, for example, at least one or a few vehicles to be allowed to be charged even if electricity prices and/or power demands are very high. It may also be possible in some embodiments to take into account a role or priority variable for an electric vehicle to determine if it should be allowed to be charged. For example, at a hospital site, recharging an electric ambulance may be prioritized over a laundry van. At a construction site, recharging an unloading vehicle may be prioritised over a hospitality truck. A vehicle with a refrigerated container attached, where the refrigerated container uses the vehicle battery system for power or which has a battery system which is also charged when the vehicle’s battery system is charged may be priorities over a vehicle without any attached vehicle accessories for example.
  • Figure 11A shows an example of a server 102 which comprise suitable processors and memory as described above in the context of Figure 6, but which implements an embodiment of method 600 shown in Figure 10A.
  • the computer code 510 comprise one or more modules or circuitry M602 or M604 which when executed cause an embodiment of method 600 to be implemented by server 102.
  • module M602 may be configured to implement step S602
  • module M604 may be configured to implement S604 shown in Figure 10A and described herein above.
  • Figure 11B shows an example of a server 102 which comprise suitable processors and memory as described above in the context of Figure 6, but which implements an embodiment of the charging policy using method 700 as shown in Figure 10B.
  • the computer code 510 comprise one or more modules or circuitry M702, M704, M706, and M708 which when executed cause an embodiment of method 700 to be implemented by server 102.
  • module M702 may be configured to implement step S702
  • module M704 may be configured to implement S704
  • module M706 may be configured to implement S704
  • module M708 may be configured to implement S708 shown in Figure 10B and described herein above.
  • the functions or steps noted in the blocks can occur out of the order noted in the operational illustrations.
  • two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality/acts involved.
  • the functions or steps noted in the blocks can according to some aspects of the disclosure be executed continuously in a loop.
  • the description of the example embodiments provided herein have been presented for the purposes of illustration. The description is not intended to be exhaustive or to limit example embodiments to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired from practice of various alternatives to the provided embodiments.
  • any reference signs do not limit the scope of the claims, that the example embodiments may be implemented at least in part by means of both hardware and software, and that several “means”, “units” or “devices” may be represented by the same item of hardware.
  • the various example embodiments described herein are described in the general context of methods, and may refer to elements, functions, steps or processes, one or more or all of which may be implemented in one aspect by a computer program product, embodied in a computer- readable medium, including computer-executable instructions, such as program code, executed by computers in networked environments.
  • a computer-readable medium may include removable and non-removable storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory, RAM), which may be static RAM, SRAM, or dynamic RAM, DRAM.
  • ROM may be programmable ROM, PROM, or EPROM, erasable programmable ROM, or electrically erasable programmable ROM, EEPROM.
  • Suitable storage components for memory may be integrated as chips into a printed circuit board or other substrate connected with one or more processors or processing modules, or provided as removable components, for example, by flash memory (also known as USB sticks), compact discs (CDs), digital versatile discs (DVD), and any other suitable forms of memory.
  • memory may also be distributed over a various forms of memory and storage components, and may be provided remotely on a server or servers, such as may be provided by a cloud-based storage solution.
  • program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
  • Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes.
  • any suitable device readable and/or writeable medium examples of which include, but are not limited to: any form of volatile or non-volatile computer readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory
  • Memory may store any suitable instructions, data or information, including a computer program, software, an application including one or more of logic, rules, code, tables, etc. and/or other instructions capable of being executed by processing circuitry and, utilized by the apparatus in whatever form of electronic apparatus. Memory may be used to store any calculations made by processing circuitry and/or any data received via a user or communications or other type of data interface. In some embodiments, processing circuitry and memory are integrated. Memory may be also dispersed amongst one or more system or apparatus components. For example, memory may comprises a plurality of different memory modules, including modules located on other network nodes in some embodiments. [000205] In the drawings and specification, there have been disclosed exemplary aspects of the disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Power Engineering (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Sustainable Development (AREA)
  • Sustainable Energy (AREA)
  • Charge And Discharge Circuits For Batteries Or The Like (AREA)
  • Electric Propulsion And Braking For Vehicles (AREA)

Abstract

La présente invention concerne un système de gestion de charge de véhicule de flotte pour une flotte comprenant une pluralité de véhicules électriques 100a, b, c qui comprend un appareil 102, par exemple, un serveur (102) conçu pour communiquer avec chaque véhicule de la flotte de véhicules, une pluralité de commutateurs logiciels (104a, b, c) et une pluralité de chargeurs de batterie (106a, b, c) conçus pour être connectés à une alimentation électrique de réseau (110, 120). Chacun des chargeurs de batterie est conçu pour fournir une alimentation électrique de charge à un véhicule électrique connecté (100a, b, c) de la pluralité de véhicules (100a, b, c) par l'intermédiaire d'un commutateur logiciel respectif parmi les commutateurs logiciels (104a, b, c) sur la base d'un modèle de charge à auto-apprentissage, le modèle de charge à auto-apprentissage étant conçu pour minimiser un coût financier et/ou une demande de puissance de mise en œuvre d'une variable de commande de politique de connexion pour charger la flotte de véhicules.
PCT/EP2022/078734 2022-10-14 2022-10-14 Gestion intelligente de puissance de réseau pour charge de véhicule de flotte WO2024078728A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2022/078734 WO2024078728A1 (fr) 2022-10-14 2022-10-14 Gestion intelligente de puissance de réseau pour charge de véhicule de flotte

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2022/078734 WO2024078728A1 (fr) 2022-10-14 2022-10-14 Gestion intelligente de puissance de réseau pour charge de véhicule de flotte

Publications (1)

Publication Number Publication Date
WO2024078728A1 true WO2024078728A1 (fr) 2024-04-18

Family

ID=84360639

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2022/078734 WO2024078728A1 (fr) 2022-10-14 2022-10-14 Gestion intelligente de puissance de réseau pour charge de véhicule de flotte

Country Status (1)

Country Link
WO (1) WO2024078728A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160347195A1 (en) * 2006-08-10 2016-12-01 V2Green, Inc. Power aggregation system for distributed electric resources
US20200254897A1 (en) * 2019-02-08 2020-08-13 Ford Global Technologies, Llc System and method for battery-electric vehicle fleet charging
US20210086647A1 (en) * 2019-09-20 2021-03-25 AMPLY Power, Inc. Real-time electric vehicle fleet management
US20210170903A1 (en) * 2019-12-09 2021-06-10 Toyota Jidosha Kabushiki Kaisha Power management system and server
US20210221247A1 (en) * 2018-06-22 2021-07-22 Moixa Energy Holdings Limited Systems for machine learning, optimising and managing local multi-asset flexibility of distributed energy storage resources
US20220188946A1 (en) * 2020-12-04 2022-06-16 Totalenergies Se Customer-centric method and system for pricing options and pricing/charging co-optimization at multiple plug-in electric vehicle charging stations

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160347195A1 (en) * 2006-08-10 2016-12-01 V2Green, Inc. Power aggregation system for distributed electric resources
US20210221247A1 (en) * 2018-06-22 2021-07-22 Moixa Energy Holdings Limited Systems for machine learning, optimising and managing local multi-asset flexibility of distributed energy storage resources
US20200254897A1 (en) * 2019-02-08 2020-08-13 Ford Global Technologies, Llc System and method for battery-electric vehicle fleet charging
US20210086647A1 (en) * 2019-09-20 2021-03-25 AMPLY Power, Inc. Real-time electric vehicle fleet management
US20210170903A1 (en) * 2019-12-09 2021-06-10 Toyota Jidosha Kabushiki Kaisha Power management system and server
US20220188946A1 (en) * 2020-12-04 2022-06-16 Totalenergies Se Customer-centric method and system for pricing options and pricing/charging co-optimization at multiple plug-in electric vehicle charging stations

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
A. O. ERICKK. A. FOLLY: "IEEE Congress on Evolutionary Computation", 2020, article "Power Flow Management in Electric Vehicles Charging Station Using Reinforcement Learning", pages: 1 - 8

Similar Documents

Publication Publication Date Title
Ko et al. Mobility-aware vehicle-to-grid control algorithm in microgrids
US9489701B2 (en) Adaptive energy management system
Shi et al. Real-time vehicle-to-grid control algorithm under price uncertainty
CA2772247C (fr) Systeme et methode permettant la planification de charge optimale de recharge de vehicule electrique
JPWO2018084152A1 (ja) 蓄電装置、輸送機器及び制御方法
Huang et al. Residential energy system control and management using adaptive dynamic programming
CN105453362A (zh) 电力供需调整系统和电力供需调整方法
US20230347778A1 (en) Delivery of stored electrical energy from generation sources to nano-grid systems
Ebadi et al. Techno-economic evaluation of transportable battery energy storage in robust day-ahead scheduling of integrated power and railway transportation networks
Ovalle et al. Optimal management and integration of electric vehicles to the grid: Dynamic programming and game theory approach
CN114801825A (zh) 针对动力电池的充电过程运行机动车的方法以及机动车
Mateen et al. Discrete stochastic control for energy management with photovoltaic electric vehicle charging station
KR20150022732A (ko) 전력계통 운영 보조서비스를 위한 전기자동차 충전개소의 참여용량 제어방법 및 시스템
US20230307918A1 (en) System and method for bi-directional direct current charging in electric vehicle supply equipment
CN113165547A (zh) 用于对一队电动车辆的充电进行优化管理的方法
Alfaverh et al. Electrical vehicle grid integration for demand response in distribution networks using reinforcement learning
EP3433828A1 (fr) Procédé et système d'agrégation dynamique d'une flotte d'unités d'alimentation permettant de fournir une régulation de fréquence d'un système d'alimentation
Yi et al. Optimal energy management strategy for smart home with electric vehicle
WO2024078728A1 (fr) Gestion intelligente de puissance de réseau pour charge de véhicule de flotte
Chen et al. Reinforcement learning for smart charging of electric buses in smart grid
KR101581684B1 (ko) 에너지 저장장치의 방전심도 제어를 통한 충방전 스케줄링 장치 및 방법
WO2024078727A1 (fr) Gestion de puissance intelligente à des fins de charge de batterie
Ovalle et al. Escort evolutionary game dynamics application on a distribution system with PV, BSS and EVs
CN114765380A (zh) 充放电控制装置
Jalilzadeh Hamidi Distributed charging management of electric vehicles in smart microgrids

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22808977

Country of ref document: EP

Kind code of ref document: A1