WO2024078727A1

WO2024078727A1 - Intelligent power management for battery charging

Info

Publication number: WO2024078727A1
Application number: PCT/EP2022/078719
Authority: WO
Inventors: Jonas Hellgren
Original assignee: Volvo Autonomous Solutions AB
Priority date: 2022-10-14
Filing date: 2022-10-14
Publication date: 2024-04-18

Abstract

A computer-implemented method (200) for charging an electric vehicle at a charging power source and related aspects involve detecting a current state of energy of a battery of a vehicle (S202), determining a target charged state of energy of the battery at a future point in time (S204); and determining a point in time when power should be provided to charge the vehicle from the charging power source based on the detected current state of energy of the battery and the determined target charged state of energy of the battery (S206) so that the future point in time is a predicted end of a charging time period, wherein the future point in time is determined using a dynamic vehicle battery charging model configured to determine the target charged battery energy state based on a predicted user demand for power from the charged vehicle at the end of the charging period and at least one charging constraint for the charging power source.

Description

INTELLIGENT POWER MANAGEMENT FOR BATTERY CHARGING [0001] The present disclosure relates to a computer-implemented method for power management when charging an electric vehicle at a charging power source and to various related aspects. In particular, but not exclusively, the disclosed technology relates to such a computer-implemented method which allows battery charging to be performed in a way which provides intelligent power management. [0002] In particular, but not exclusively the disclosed technology relates to a self-learning method of grid power management for charging an electric vehicle which seeks to control in an adaptive and optimal manner when an electric vehicle should be charged or not charged. [0003] The method is particularly useful for electric vehicles which are automated in some way, for example, they may be autonomous or semi-autonomous and/or remote-controlled. The disclosed invention will be described mainly with respect to vehicles, however, such vehicles may include heavy-duty vehicles, such as semi-trailer vehicles and trucks as well as other types of vehicles such as cars. Electric vehicles may also refer to electric vehicles and any electrically connected vehicle accessories such as trailers and/or containers and the like. Such vehicle accessories may, in some embodiments, be able to recharge via an electric vehicle’s charging system and/or may have independently re-chargeable battery systems. BACKGROUND [0004] The disclosed technology seeks to mitigate, obviate, alleviate, or eliminate various issues known in the art which affect how to manage power consumption when charging an electric vehicle to avoid drawing power at times where power availability is lower due to demand. For example, in times of peak demand, to deter some customers from using electrical power distributed from a power grid, tariffs set for power consumption from the grid may be punitive. It is therefore helpful if power can be managed to avoid such periods of peak demand. [0005] Determining when to charge an electric vehicles, EVs, in a residential charging environment, may also be complicated by the maximum amp of the residential supply and the amount of power it will take to charge the electric vehicle as other devices such as a dishwasher or washing machine etc., may also need to draw power at the same time. Even if a grid power supply is supplemented or replaced by a local power generation source, for example by a solar source, there will be limits as to how much power may be consumed at the same time. Residential single phase systems are often limited to a certain number of KWatts per hour, and if the demand for power exceeds this then a three-phase supply may be needed. This requires additional infrastructure to be installed as well as having cost. Accordingly, it is important to be able to control how much power is used and when it is used to charge an electric vehicle. [0006] A. O. Erick and K. A. Folly disclose in their paper "Power Flow Management in Electric Vehicles Charging Station Using Reinforcement Learning," 2020 IEEE Congress on Evolutionary Computation (CEC), 2020, pp.1-8, doi: 10.1109/CEC48606.2020.9185652 describe an example of optimal power flow management problem in an electric vehicle charging station. The charging station is powered by solar PV and is tied to the grid and a battery storage system through necessary power conversion interfaces for DC fast charging. The optimal power management problem for EV charging is solved via reinforcement learning (RL). Unlike classical optimization methods such as dynamic programming, linear programming (LP) and mixed- integer linear programming which are limited in handling stochastic problems adequately and are slow due to the curse of dimensionality when used for large dynamic problems, RL does not have to iterate for every time step as learning can be done completely offline and optimal solutions saved in a lookup table, from which optimal control actions can be retrieved almost instantaneously. [0007] Electric vehicle charging stations known in the art however do not take into account issues such as peak demand and dynamic pricing where the price per unit of electricity, for example, the price per kWh varies during a given time period, such as over the course of 24 hours or weekly or seasonally, or as a result of environmental conditions changing such as wind and solar radiation, all of which may affect the amount of power available to be delivered to a vehicle. SUMMARY STATEMENTS [0008] Whilst the invention is defined by the accompanying claims, various aspects of the disclosed technology including the claimed technology are set out in this summary section with examples of some preferred embodiments and indications of possible technical benefits. [0009] A first aspect of the disclosed technology comprises a computer-implemented method for charging an electric vehicle at a charging power source, the method comprising: detecting a current state of energy of a battery of a vehicle, determining a target charged state of energy of the battery at a future point in time, and determining a point in time when power should be provided to charge the vehicle from the charging power source based on the detected current state of energy of the battery and the determined target charged state of energy of the battery so that the future point in time is a predicted end of a charging time period, wherein the future point in time is determined using a vehicle battery charging model configured to determine the target charged battery energy state based on a predicted user demand for power from the charged vehicle at the end of the charging period and at least one charging constraint for the charging power source. [00010] In some embodiments, the method according to the first aspect comprises training the vehicle battery charging model to map input to the model comprising a set of power load variables of the charging power source to an output representing a charging action to charge the battery or a non-charging action to not charge the battery by rewarding for long-term and short- term consequences of the charging or non-charging actions. [00011] The battery charging model may be a dynamically trained reinforcement algorithm based charging model which uses cumulative rewards in some embodiments. [00012] In some embodiments, the first aspect comprises computer-implemented method of power management for charging an electric vehicle, the method comprising determining a current state of energy of a battery of a vehicle determining a target desired charged state of energy of the battery at the end of a charging period, determining if power should be provided to charge the vehicle from a charging power source using a charging model for the power source based on the current state of energy of the battery and the determined target charged state of energy of the battery, wherein the dynamic charging model is trained to map input comprising a set of power load variables of the charging power source to an output comprising an action to charge the vehicle battery or not, wherein the model is trained using rewards for long-term consequences of the action and rewards for short-term consequences of the action, and wherein the trained charging model determines a target charged battery energy state based on a predicted user demand for power from the charging power source at a predicted end of the charging period based on a cost of charging constraint. [00013] Advantageously the disclosed method may be implemented using a soft-switch so that a battery may be connected but only charged a subsequent point in time and still achieve a desired end state of energy. [00014] In some embodiments, the battery is determined to start charging at the future point in time, based on the determined current state of energy of the battery and a determined charging action of the dynamic vehicle battery charging model, configuring the external charging power supply to automatically provide power to charge the vehicle by actuating a soft switch or relay (S208). [00015] Advantageously, the method does not need to perform a forecast for a supply load profile and day-ahead tariff. [00016] Advantageously, the method may learn over time the optimal time to recharge a vehicle based on its use history and charging history so that its down-time for recharging is minimised. Consequently, the method has the potential to improve the overall efficiency of use of an electric vehicle as it improves likelihood that the electric vehicle will have at least a minimum battery state of energy for a predicted desired use at a particular point in time. The technology may also improve the charging performance of the battery of an electric vehicle as the soft-switch can also turn off power which may prevent over-charging, and so it may help reduce battery degradation over time and/or assist with maintaining battery capacity closer to the maximum possible by better managing the charging cycles. The soft-switch may also be configured to disconnect a battery from charging using grid power due to peak power demands or responsive to a tariff price change increase. By reducing power consumption during peak power demands, when tariffs are more expensive, the disclosed technology also helps reduce the financial cost of charging electric vehicle batteries. [00017] In other words, the disclosed charging models are configured in some embodiments to allow for energy consumption to begin and end at multiple different times during a charging cycle without the need for direct human intervention so as to manage demand for grid power and/or any dynamic electrical tariff pricing changes. For example, if the price per kWh of electricity is dependent on environmental factors such as the windspeed and/or solar radiation, increases in either or both may result in locally at that charging station and/or regionally within a grid, the cost of energy changing at various times which may not be intuitive for a human user to know when to plug their vehicle in. Also, in some countries, lower prices are charged for each kWh of electricity consumed during off-peak demand periods than are charged during peak demand periods. This may also influence when an electric vehicle is charged or not. [00018] In some embodiments, based on the predicted future point in time for achieving the determined target charged battery state, the method further comprises, responsive to determining that power should start to be provided to charge the vehicle to achieve the predicted target charged battery state by the determined future time, causing an alert to be provided to a user to connect the vehicle to a power supply if the vehicle is not already connected (S210). [00019] In some embodiments, at least one charging constraint is a charge or not condition based on an objective to minimize long term energy costs. [00020] In some embodiments, at least one charging constraint is a cost of charging constraint and/or an available power constraint. For example, if power cuts occur when demand exceeds a limit, this may be taken into account in the form of a charging constraint. [00021] In some embodiments, the set of power load variables of the external power source includes a power load variable of an external power supply to the charging power source, wherein the external power supply is connected to one or more other power-consuming devices consuming power in a charging period. [00022] In some embodiments, the charging constraint comprises a maximum available power constraint for the external power supply to provide power to the charging power supply. [00023] In some embodiments, the power-consuming devices include one or more devices which are not battery operated. [00024] In some embodiments, the charging model is trained to determine, for each of a plurality of charging periods, electricity cost and a probable risk of a battery state of energy not meeting a target charged battery energy state at the end of that charging period. In some embodiments, In some embodiments, the method further comprises varying the amount of charging power used by the charging power supply to charge the vehicle battery based on the current vehicle battery state of energy. In some embodiments, the method further comprises obtaining data generated by implementing the dynamic vehicle battery charging model at the charging power supply, using the gathered data as a training data set for training the dynamic battery vehicle charging model, wherein the gathered data comprises, for each charging period, data representing the start time, the end time, the initial battery charge state, the end battery charge set, the power consumed charging the battery and the cost of the power consumed charging the battery to the charging model, and re-training the trained dynamic battery vehicle charging model by inputting the gathered data into the trained model. [00025] Another aspect of the disclosed technology comprises a method of training a charging model using reinforcement learning, the charging model being configured for use in a method of the first aspect or any one of its embodiments disclosed herein, the trained model being configured to determine when to charge the vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods, the method comprising inputting training data to the charging policy model, and iteratively processing the input training data to provide iterative output representing a charging cost, wherein each iteration comprises comparing a charging cost with a target charging cost or with a charging cost from a previous iteration and maximising cumulative rewards for reducing the charging cost to achieve the target battery charge state. [00026] In some embodiments, each reward, r, of the maximised cumulative rewards, can be represented for each charging period as either a charge power consumption per unit time multiplied by a duration of time over which charge power is consumed multiplied by the energy consumption cost per unit of power in the case where the end of the charging period is reached and the target battery energy state is reached; or, if the target battery energy state is not reached at the end of the charging period, a total energy consumption cost for the duration of time the battery was charged plus a penalty term comprising the energy deficiency at the end of the charging period multiplies by a penalty constant. [00027] In some embodiments, wherein each reward r of the maximised cumulative rewards is determined using a reward function based on a charge power consumption per unit time, a duration of time over which charge power is consumed, a battery energy deficiency comprising a difference between a target state of battery energy and a current state of battery energy, an energy consumption cost, and wherein the reward function indicates if an end of a charging episode is reached or not, and includes a penalty term defining how a violation from a target state of energy is to be penalised. [00028] A third aspect relates to an apparatus comprising a computer-program product comprising a set of instructions stored in a memory and comprising one or more modules or circuitry which, when loaded from the memory for execution by one or more processors or processing circuitry of the apparatus cause the apparatus to implement a method according to the first and/or second aspects or any of their embodiments disclosed herein. [00029] In some embodiments, the apparatus is configured to be connected to a grid power supply and comprises a charging power supply outlet configured to be connected to an actuatable a soft-switch 104, wherein, when an electric vehicle 100 is connected to the apparatus, the apparatus is configured to actuate the soft-switch to control a supply of electrical power to at least start charging a battery of the connected electric vehicle based on the apparatus implementing a method according to the first aspect or any one of its embodiments disclosed herein to determine an action to charge a connected vehicle should be performed. [00030] In some embodiments, wherein actuating the soft-switch 104 also controls the supply of electrical power to stop charging a battery of the connected electric vehicle when the target charged state of energy of the battery is reached. [00031] In some embodiments, wherein the apparatus comprises a battery charger. [00032] In some embodiments of the above first, second and third aspects at least, the electric vehicle comprises a heavy electric vehicle. [00033] In some embodiments of the above first, second and third aspects at least, the electric vehicle includes an electrically connected vehicle accessory have a battery system capable of being charged via the electric vehicle. [00034] In some embodiments of the above first, second and third aspects at least, the electric vehicle comprises an autonomous, semi-autonomous or remotely controlled electric vehicle. [00035] In some embodiments, the apparatus may comprise an electric vehicle configured to self-regulate its own charging. For example, the electric vehicle may use wireless communications to communicate with a remote server 102 configured to implement a method according to the first aspect, wherein the server communicates to the electric vehicle control signals to actuate the soft-switch in some embodiments. [00036] Another aspect of the invention comprises a computer-readable storage medium comprising computer-program code which, when executed by one or more processors or processing circuitry of an apparatus, causes the apparatus to implement a method according to the first aspect. [00037] A computer program carrier carrying a computer program comprising computer- program code, which, when loaded from the computer program carrier and executed by one or more processors or processing circuitry of an apparatus causes the apparatus to implement a method according to the first aspect, wherein the computer program carrier is one of an electronic signal, optical signal, radio signal or computer-readable storage medium. [00038] A computer program product comprising computer-code which when loaded from memory and executed by one or more processors of a control circuit of an apparatus causes the vehicle to implement a method according to the first aspect. [00039] Computer code for causing an apparatus to perform the method of the first aspect when said computer code is loaded from memory and run on one or more processors or on processing circuitry of a an apparatus. [00040] The disclosed aspects and embodiments may be combined with each other in any suitable manner which would be apparent to someone of ordinary skill in the art. BRIEF DESCRIPTION OF THE DRAWINGS [00041] Some embodiments of the disclosed technology are described below with reference to the accompanying drawings which are by way of example only and in which: [00042] Figures 1A and 1B schematically illustrate example power cost models over a 24 hour period; [00043] Figure 1C schematically illustrates a power management system for charging an electric vehicle according to some embodiments of the disclosed technology; [00044] Figures 2A and 2B schematically illustrate example charging episodes according to some embodiments of the disclosed technology; [00045] Figure 3A schematically illustrates a charging model according to some embodiments of the disclosed technology; [00046] Figure 3B schematically illustrates a charging policy based on a charging model to some embodiments of the disclosed technology; [00047] Figure 4A schematically illustrates an example of a method of power management for charging an electric vehicle according to some embodiments of the disclosed technology; [00048] Figure 4B schematically illustrates an example method of training a charging model using reinforcement learning according to some embodiments of the disclosed technology; [00049] Figure 5 illustrates schematically an example of an apparatus configured to perform a method of power management for charging an electric vehicle according to some embodiments of the disclosed technology; [00050] Figure 6 illustrates schematically an example of an apparatus configured to train a charging model according to some embodiments of the disclosed technology; and [00051] Figure 7 shows an example data flow when training a reinforcement learning charging model according to some embodiments of the disclosed technology. DETAILED DESCRIPTION [00052] Aspects of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. The apparatus and method disclosed herein can, however, be realized in many different forms and should not be construed as being limited to the aspects set forth herein. Steps, whether explicitly referred to a such or if implicit, may be re- ordered or omitted if not essential to some of the disclosed embodiments. Like numbers in the drawings refer to like elements throughout. [00053] The terminology used herein is for the purpose of describing particular aspects of the disclosure only, and is not intended to limit the disclosed technology embodiments described herein. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. [00054] Electric vehicles require battery charging and usually to charge their batteries they are periodically connected to a mains electricity supply, often referred to as “the grid” which is provided using a power distribution network from an electrical power station which may, for example, comprise a hydro-electric, wind, nuclear or other type of electrical energy source. The demand for electrical energy from a mains electricity supply has well-known peaks and lows, and electrical energy providers often try to incentivise the use of electrical energy at off-peak times by offering favourable tariffs. So, for example, 1 kWh of electricity consumed during a peak demand time period may cost twice as much as 1kWh of electricity consumed during an off-peak demand time period. [00055] Electric vehicle batteries may be configured to store several 10s or more of kWhs of electrical energy. Typically battery charging is performed over long time-periods last several hours and in order to minimize the operating cost of the vehicle, it is advantageous if the cost of any electric energy bought from the grid is as cheap as possible. [00056] Figures 1A and 1B schematically illustrate example power pricing models over a 24 hour period for charging an electric vehicle using a mains power accessed via a battery charger 104. As shown, the cost of charging the vehicle varies over a 24 hour period. Electric vehicles, particularly when the vehicles are heavy-duty vehicles which may have larger battery capacity and require higher voltages than smaller vehicles, may take several hours to fully charge depending on the battery power capacity and the rate at which the battery charger can recharge the battery. [00057] As shown in Figure 1A, a vehicle which begins charging at time T0 and finishes charging at time T_end will be being charged over a period of time that crosses a plurality of charging tariffs where a 1kWh costs between cost #1, cost #2, and a final cost #3. These costs reflect in part the supply demand on the power distribution network which is illustrated schematically in Figure 1B. [00058] In order to improve the management of the power supply so that the cost of charging a vehicle battery using the power supply is minimised, it is advantageous to be able to control when the vehicle starts to charge, i.e. to control time T₀ and also to limit the power used to charge the battery, which may be by limiting the time taken overall or just the time taken during periods of time when the charges are not the lowest. [00059] Determining when the optimum time is to charge a vehicle however may not be straight-forward. For example, grid-based pricing for electricity may vary geographically and may be adjusted depending on local micro-generation. Costs may also vary between cities, regions, and countries. [00060] Figure 1A shows an example plot of an electrical energy price model where the price of electricity is shown a function of time. In the example shown, it is more expensive to use electricity at evenings when cost #3 applies and demand is high (as shown in Figure 1B) than in the early morning when cost #1 applies. Figures 1A and 1B accordingly show an example of a pricing model where a customer gets “punished” for using power during peak periods of demand. The power during a period is tracked and the electricity price is related to the peak power. This incentives householders and businesses for example to use electricity during off- peak rates and to avoid during peak-rates charging a vehicle battery. Moreover, it is also advantageous to avoid battery charging when there are other demands on the power being supplied, for example, made by machinery or household appliances. Sometimes, however, in order to be certain a vehicle battery will be fully charged or sufficiently charged by a certain time, a user may connect a vehicle to a battery charger regardless of the demand or cost at a particular time of day. This is not desirable as it increases the demand for power to be delivered over the power network infrastructure during a peak rate as well as increasing the cost of the power supplied to charge the vehicle battery. [00061] Figure 1C schematically illustrates a power management system for charging an electric vehicle according to some embodiments of the disclosed technology which seeks to control if a vehicle should be charged or not in an adaptive and optimal manner. As shown in Figure 1C, a vehicle 100, for example, heavy-duty vehicle, may be connected to a battery charger 106 via a soft-switch 104 which may be controlled either by the battery charger 106 or remotely by a server 102 configured to control operation of the battery charger and/or the soft-switch. As shown in Figure 1C, the soft-switch may comprise a relay which enables a battery of the vehicle 100 to be connected and/or disconnected from the electrical power supply provided by the battery charger. In some embodiments, the soft-switch is controlled using switch control software which sets the relay state of the switch according to one or more inputs the switch control software receives. The relay of the soft-switch may according be fairly simple and not require any sophisticated power electronics. Advantageously, this results in a soft-switch which may require little maintenance. [00062] The disclosed technology uses a self-learning battery charging and power management system that seeks to manage power demands and/or grid electricity costs when charging a battery of an electric vehicle to find the optimal balance between energy cost and the risk of not ending up in a satisfying battery state of energy in a given charging episode. The self- learning battery charging and power management system is implemented so that there is a minimum amount of user input and/or user awareness of the decision process the management system performs knowledge and so that during a charging episode, is a trade of between the electricity cost and the risk of not ending up in a satisfying battery state of energy. [00063] Figures 2A and 2B schematically illustrate example charging episodes according to some embodiments of the disclosed technology where power is illustrated as a solid line and the battery state of energy SoE is shown as a dashed line. For both illustrated charging episodes, the target SoE of the battery is 100%. In Figure 2A, however, the charging policy implemented was too risk adverse, in other words, it was too careful and overly prioritized short term energy cost savings and as a result, was exposed to higher energy costs later in order to reach the target SoE by the target future point in time, which is 8:00am in the example illustrated. In contrast, in the example charging episode shown in Figure 2B, a better balance between electricity cost and the risk of not ending up in a satisfying battery state of energy is achieved. [00064] As used herein, the battery state of energy, in other words, the battery SoE is a normalised number which is 100% for a fully charged battery. The SoE_error is the target SoE minus the actual SoE. “action” is a Boolean decision taken by a charging policy which is true if charging is recommended and t_norm is normalized time, the spend charging time divided by the time in the day of the charging. [00065] In some embodiments, the charging apparatus 106 uses a non-rule-based charge policy, for example a charge policy configured to use one or more pattern recognition techniques or machine learning algorithms to determine an action to perform, for example, an action which determines when charging should start and/or end in order to charge a battery to a target SoE at a future point in time. [00066] In some embodiments, the policy maps a set of variables, for example ^^^_^^^^^ and ^_^^^^, to an optimal action comprising whether the vehicle charger shall receive charging power or not. The charging power is referred to as grid connected power or grid power in embodiments where electrical energy is received from a power distribution network or grid mains power supply. In some embodiments, however, the charging power may be partly or wholly indirectly grid powered if a storage battery configured to be grid powered for back-up purposes is used. In some embodiments, such a battery may also be connected to a local power generation source such as photo-voltaic solar panels or a wind turbine. [00067] Figure 3A illustrates schematically an example embodiment of such a charge policy. In Figure 3A, input to the charging model is processed by the charging model and the model generates output in the form of an action to charge a battery or to not charge the battery in a way that optimizes a reward function, r, and which updates a state transition function, ^̅. [00068] In some embodiments, the reward function uses cumulative rewards, for example, as follows:

The terms used in the above reward function, r, are defined in the table below as follows: Term Unit Description ^_^^ (kW) Charge power, typically 2-3 kW in a household. ^^^^_{^^^^ ^^^^^^} (Euro/kWh) The cost related to energy consumption ^^^^^^^^⁽^̅⁾ A function true if end of charging episode is reached, i.e. if ^_^^^^ ≥ 1 ^^^^_{^^^^ ^^^^} (varies) This term depends on the cost model. For example, it is related to peak power if the model is according the right panel in Figure 1. For the price model in the left panel it is zero. ^̅ A set of variables used by the charging policy which are derived from ^̅. These may be referred also to as features in a machine- learning context. ^̅ State variables. CP (Euro) Penalty constant, defines how a violation from target SoE shall be penalized. ∆t (hour) Time step [00069] In some embodiments, the transition function of the charging model which expresses how a state ^̅ is to be updated is defined by: 1) ^̅ ← ^(^̅, ^^^^^^) For the specific problem of charging it includes, for example, the relation 2) ^^^ ← ^^^ + ∆^^⁄ ^_^^^^^^^ [00070] In some embodiments, using the cumulative reward and transition functions as defined above in a suitable reinforcement learning algorithm the value of an action in a particular state can be used. Reinforcement learning involves an agent, a set of states, S, and a set of actions A per state. By performing an action a ∈A, the agent transitions from state to state, and the agent is rewarded each time an action is executed in a particular state, where the reward is a numerical score. [00071] Some embodiments of the charging model may use a reinforcement learning algorithm where the agent seeks to maximize its total cumulative reward by adding the maximum potential reward attainable from future states to its reward for achieving its current state, where the potential reward is a weighted sum of expected values for the rewards in all future steps starting from the current step. This influences the current action by the potential future reward. So, for example, using such a charging model could learn that, even if initially for a given set of input variables where the reinforcement learning algorithm output an action to charge the battery starting from a time t_x which resulted in a negative reward, for example, as the target SoE is not reached at the end of a charging period, that repeatedly charging the battery starting at that time tx results in in the long term that starting at time tx is better than at other start times, as the over-all reward is better with the start time tx. [00072] In some embodiments, a model-free reinforcement learning algorithm may be used such as, for example, a Q-learning reinforcement learning algorithm is used which does not require a model of an environment and which can handle stochastic transitions and rewards without requiring adaptations. The “Q” being learnt is the reward function computed by the learning algorithm for an action in a given state. A Q-learning reinforcement learning algorithm seeks to find an optimal policy starting from a current state S for any finite Markov decision process which maximizes the expected value of the total reward over any and all successive steps, starting from a current state. In other words, by repeating possible actions for which a reward is provided a sufficient number of times, actions which result in optimum rewards can be learnt. [00073] Advantageously, embodiments of the charging model which use reinforcement learning do not have to iterate for every time step as learning can be done offline and optimal solutions stored in memory, for example, in a look-up table such as that shown in Figure or such as in a neural instantaneously. to iterating every time-step in a more energy efficient way. [00074] after a certain number of steps

factor between 0 and 1 which determines the importance of future rewards. The Q-learning algorithm has a function that calculates the quality of a state-action combination as follows: ^: ^ × ^ → ℝ Q is initialised to a value before learning begins and then each time t the agent selects an action a_t ^ A, a reward, r_t, is calculated and the agent enters a new state s_t+1, which may depend on both the previous state s_t and the selected action, and Q is updated. For example: ng ing

[00076] The “Q” which is being learnt, in other words, the reward function computed by the learning algorithm for an action in a given state, provides an indication of how good or bad it is to take a specific action. If Q(^^̅,1)>Q(^^̅,0), then it is better to take action 1, then 0, in state ^^̅. Q expresses the long-term consequence of an action, the cumulative reward, while r is the short term reward. A simple look-up table can be used as memory for holding the Q values learnt for each new state s’ after taking an action a for the tabular Q-learning algorithm. [00077] Some embodiments of the invention, however, may use a more complex Q-learning algorithm, for example, one using neural networks acting as memory may be expressed in pseudo-code as: Initialize primary network Q_θ, target network Q_θ’, replay buffer D, τ << 1 for each iteration do for each environment step do Observe state s_t and select α_t̴ π (α_t, s_t) Execute αt and observe next state st+1 reward rt = R(st,αt ) Store (st, αt, rt, st+1 ) in replay buffer D for each update step do Sample et = (st, αt, rt, st+1 )~ D Compute target Q value: Q*(s_t, α_t) ≈ r_t + γQ_θ (s_t+1,argmax_α’*Q_θ’ (s_t+1, α’ )) Perform gradient descent step on (Q*(s 2 t, αt)- Qθ(st, αt)) Update target network parameters: θ’ ←τ* θ + (1-τ) * θ’ [00078] An embodiment which uses tabular Q-learning may result in a charging policy such as that shown in Figure 3B. This shows for each state the actions performed and the result in each square or “tab” of the chart shown. For the charging policy illustrated in Figure 7, the following assumptions were made: 1) a grid with 1 hour resolution, 2) 25% SoE resolution 3) 15- hour charge episode time, 4) the pricing shown in Figure 1A which has electricity being more expensive in the first half of each charge episode, and where 5) a time step parameter ∆t corresponds to the width of a cell in the grid. [00079] In Figure 3B darker shaded squares correspond to when implementing the charging model resulted in an action to charge the vehicle, in other words a true action variable. On the y- axis is the SoE_error which is positive when the actual SoE is lower than the target SoE. If zero or negative, charging is not rewarded. In the example shown in Figure 3B the policy shows that charging is done for small SoE deviations only when electricity is cheap. [00080] By using such a policy which is based on data and facts, trial and error human intuition about when to start charging a vehicle, which could either lead to an SoE_error when the battery does not reaching its target charge state or to reaching its charge state but at a price cost which is too high is avoided. [00081] In some embodiments, to avoid a reinforcement learning algorithm testing new random actions which may result in a poor ^^^ after a charging episode, the model algorithm is trained off-line using recorded data. For example, a buffer may be provided with recorded price data, rpdb. For every charging episode, almost once per day for daily commuters using an eV, a price data trajectory is added and stored in rpdb. Until an adequate amount of data is present in rpdb, a naive policy may be used in some embodiments, for example, a policy which always recommends charging. After some price data collection has occurred, the initial training may start and a draft policy may be recommended based on the initial training. Training can be repeated so that a draft policy is recommended, maybe after 20-30 charge episodes. [00082] In some embodiments of the charging policy or model, the action is not restricted to be true or false, but rather, for example, a floating point number within a range, for example, within a normalised range of [0,1]. [00083] Figures 4A and 4B show example embodiments of how the server 102 or the battery charger apparatus 106 may implement a method which uses a charging policy model such as one of the above examples which use a Q-learning algorithm to determine when to supply charging power so that a battery is charged to a target SoE by a target future point in time, for example, by the 104 shown in Figure 1C to charge of electric vehicle with a Examples of vehicle 100 include a may be a plug-in hybrid or sole source of power for method 200 for power

charging power source 106. The method 200 may comprise detecting a current state of energy of a battery of a vehicle (S202), in other words, measuring or sensing the actual battery state in some embodiments, determining a target charged state of energy of the battery at a future point in time (S204) and determining a point in time when power should be provided to charge the vehicle from the charging power source based on the detected current state of energy of the battery and the determined target charged state of energy of the battery (S206) so that the future point in time is a predicted end of a charging time period, wherein the future point in time is determined using a dynamic vehicle battery charging model configured to determine the target charged battery energy state based from the charged vehicle at the end of the

constraint for the charging power source. [00085] Method 200 is performed by the battery charger 106 detected vehicle state

is a measured of sensed battery vehicle state which may be the soft-switch 104 in some embodiments. The future point in time may be learnt by the

of the charging model or input by a user of the vehicle to the battery charger in some embodiments. In other words, given a known future point in time when the vehicle is to be driven, the goal of the charging model is to minimise the financial cost of charging the battery so that it is ready for use with a minimum desired state of energy of the vehicle battery at that point in time. The dynamic vehicle battery charging model may use a reinforcement learning algorithm with cumulative rewards such as one of the above mentioned examples which may use tabular Q-learning or neural-network Q-learning to provide a cumulative reward for actions which charge the battery SoE to the target SoE by a particular future point in time, which may be user specified or which may learnt from historic charging cycles. [00086] In some embodiments the charging model receives a set of power load variables for the charging power source and processes these to determine an action to charge the battery or a non-charging action to not charge the battery by rewarding for long-term and short-term consequences of the charging or non-charging actions. [00087] By learning usage patterns for the electric vehicle it allows the charging system to be a self-learning system in some embodiments that minimizes grid electricity expenses for battery charging without risking the vehicle not having a sufficient amount of charge when it is needed to be used. [00088] In some embodiments, as shown in Figure 4A, the method 200 further comprises, if the model determines an action that battery is to be charged based on the determined current state of energy of the battery and the charging model, configuring the external charging power supply to automatically provide power to charge the vehicle by actuating a soft switch or relay S208. Based on the predicted target charged battery state being required at a time of day, in some embodiments, the method may further comprise, responsive to determining that power should be provided to charge the vehicle to reach the predicted target charged battery state by the required time of day, causing an alert to be provided to a user to connect the vehicle to a power supply if the vehicle is not already connected S210. In other words, to ensure that actuating the soft-switch 104 will result in the vehicle being charged from the battery charger 106.

[00089] The charging model used by method 200 may in some embodiments be trained and/or retrained by using a set of power load variables for the battery charger, a term which may also refer to any other suitable form of external power source 106 which takes grid power directly or indirectly and uses it to charge the vehicle. For example, in some embodiments the set of variables include a power load variable on the external power supply for each of one or more other power-consuming devices, for example, household appliances or machinery consuming power from the electrical installation connected receiving grid power as the battery charger 106in a charging period. The electrical installation may be the site or premises distribution board, or a local transformer sub-station, or similar local area. By modelling the power load on the electrical installation in a way that takes into account the use of other power consuming devices drawing power at the same time as the battery charger 106 is charging the battery of the electric vehicle, the maximum grid power drawn via the electrical installation can be managed and any limits not exceeded. This may help prevent overloading and/or one or more appliances, including the battery charger to failing to operate normally. The power- consuming devices may be battery-operated and connected to battery chargers connected to grid-power in some embodiments. One or more power-consuming devices may use the same battery charger 106 used to charge the battery of the vehicle 100 in some embodiments. However, the power-consuming devices may include one or more devices which are not battery operated and directly connected to the grid in some embodiments. For example, appliances such as central heating boilers, air-conditioners, washing-machines, computing equipment etc. [00090] The method 200 uses charging model which is trained to determine, for each of a plurality of charging periods, electricity cost and a probable risk of a battery state of energy not meeting a target charged battery energy state at the end of that charging period. The amount of charging power used by the charging power supply to charge the vehicle battery based on the current vehicle battery state of energy may be varied in some embodiments, for example, based on the price cost of the electricity consumed and/or responsive to other demands on the electrical installation to which the battery charger is attached and/or to other demands on grid power. For example, in periods of peak consumption, grid power may be limited to avoid cutting-out supply to other installations for example, on premises such as hospitals and the like. [00091] To account for the electricity cost varying over a plurality of charging periods, the charging model is configured to modify the reward function to disincentivise the use of power during peak periods in some embodiments. For example, the charging model vary the amount of charging power to minimise the electricity cost for charging the battery over one or more charging periods. For example, a charging model which uses Q-based learning algorithm where Q represents cumulative rewards comprising short and long term rewards based on the consequences of charging the battery meeting the cost constraints or not may be used in some embodiments. [00092] In such embodiments, the charging model which is implemented using the method 200 shown in Figure 4A is configured to use a Q-based learning algorithm configured to determine whether to charge the battery or not using cumulative rewards for reducing the charging cost whilst still achieving a target SoE by a desired future point in time. Unless trained, however, in other words, until the charging model has learnt what battery charger behaviours are likely to lead to the target SoE being reached in time, the battery charger may cause the battery to not be charged when a user wants to use the vehicle. To avoid this happening, it is useful to pre-train the charging model. Training may use historic user behaviour data obtained via the charging station 106 or another charging station and/or historic vehicle use data obtained from the vehicle or a remote data repository in some embodiments. [00093] Accordingly, in some embodiments, method 200 implements a dynamic charging model configured to be retrained from time to time using data gathered from implementing the model at a charging power supply and sends data for each charging period representing the start time, the end time, the initial battery charge state, the end battery charge set, the power consumed charging the battery and the cost of the power consumed charging the battery to the charging model S212. [00094] Figure 4B illustrates an example of how the charging model may be trained or retrained according to some embodiments of the disclosed technology in which the charging model is trained using a method 300 which implements a reinforcement learning algorithm. The training data may take the form of electrical energy pricing and demand data over a given time period such as 24 hours, sampled over a longer time period, for example, weeks or months or longer. The data shown in Figures 1A and 1B shows schematically examples of data which may be used to train the charging model. [00095] The charging model which is configured using method 300 may be used as a trained charging model in an embodiment of method 200. The trained charging model is configured to determine when to charge the vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods by inputting training data to the charging policy model S302 and iteratively processing the input training data to provide iterative output representing a charging cost S304. Each iteration performed comprises: comparing a charging cost with a target charging cost or with a charging cost from a previous iteration S306 and maximising cumulative rewards for reducing the charging cost S308 to achieve the target battery charge state. [00096] Each reward, r, of the maximised cumulative rewards, can be represented for each charging period as either a charge power consumption per unit time multiplied by a duration of time over which charge power is consumed multiplied by the energy consumption cost per unit of power in the case where the end of the charging period is reached and the target battery energy state is reached or, if the target battery energy state is not reached at the end of the charging period, by a total energy consumption cost for the duration of time the battery was charged plus a penalty term comprising the energy deficiency at the end of the charging period multiplies by a penalty constant. [00097] In some embodiments, each reward r of the maximised cumulative rewards can be represented by a reward function comprising the following expressions or computationally equivalent expressions: ∆^ ∙ ^^^^_{^^^^ ^^^} (^^^^^^^^⁽^̅⁾) [00098] ^ = − ^ _^^^ ^^^^_{^^^^ ^^^^} + ^^^^^^^(^̅) (^^^^) [00099] where ^ ∆^ = _^^ ∙ ∆^ (^^^^^^ = ^^^^) [000100] ^ 0 (^^^^) |^^^_^^^^^ ^| ∙ ^^ (^^^_^^^ > 0) [000101] ^^^^^^^⁽^̅⁾ = ^ _^^ 0 (^^^^) [000102] where P_ch is the charge power consumption per unit time, Δt is the duration of time over which charge power is consumed, SoE_error represents an energy deficiency defined as a difference between a target state of energy, SoE, of the battery and a current state of energy,, where costelec energy is the energy consumption cost, where notEnded(^̅) is a function returning true if the end of a charging episode is reached and tnorm >=1, and CP is a penalty constant which defines how a violation from target SoE is penalised. The state of battery energy may be normalised to 100% for a fully charged battery. [000103] In some embodiments, as shown in Figure 4B by the S302-S304-S306-S308-S310 dashed lines, method 300 may comprise a method 300a which initially trains the self-learning charging model off-line using training data comprising price data trajectories for a plurality of historic charging episodes/events. The method 300a may also, after each battery charging episode, gather as new training input data the input data to the charging model associated with the charging episodes and the charging model data associated with each charging episode ending which is output in S310, and, after a number of battery charging episodes, updating the training data S301 and retraining the charging model using method 300b. [000104] In charging model may be implementing using a computer-program product in some embodiments by an apparatus comprising the server 102 and/or battery charger 106. For example, in some embodiments, the computer-program product may comprise a set of instructions stored in a memory and comprising one or more modules or circuitry which, when loaded from the memory for execution by one or more processors or processing circuitry cause a charging model trained using an embodiment of one or more of the methods 300, for example as shown in Figure 4B, and then once trained, implemented in an embodiment of method 200 as shown in Figure 4A. In some embodiments, the trained charging model includes a set of power load variables for a charging power supply which comprise the following variables: a variable representing a current time of day, a variable representing a charging tariff for power drawn based on the current time of day, a variable representing a target charged battery energy state, and a variable representing a current energy state of the battery. [000105] In some embodiments, executing the instructions causes the charging model to map a current battery state of energy, a target battery energy state, and the duration of battery charging divided by the time of day spent charging to determine a charging cost to reach the target battery energy state. The charging model may comprise a self-learning model trained using a reinforcement machine learning to determine when to charge an electric vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods. For example, in some embodiments, the reinforcement learning model is configured to maximise cumulative rewards for reducing the charging cost, and may comprise a Q-learning algorithm such as one of the example Q-learning algorithms disclosed herein above. Such a reinforcement learning model is configured to maximise cumulative rewards for reducing the charging time. [000106] In some embodiments of the disclosed technology, the charging model is a dynamic charging model which is configured to be retrained from time to time using data gathered from implementing the model at a charging power supply, such as Figures 4A and 4B show collectively. [000107] The battery charger apparatus 106 and/or server apparatus 102 comprise suitable memory, one or more processors or processing circuitry, and computer code stored in the memory, so that by executing the computer code loaded from the memory the battery charger and/or server apparatus are configured to implement a method of charging a battery using a trained learning charging model which uses a cumulative reward system according to the disclosed embodiments. [000108] In some embodiments, the computer code comprises a set of instructions stored in the memory, the set of instructions comprising one or more modules or circuitry such as Figure 5 shows schematically for the battery server 106 and Figure 6 shows schematically for the server 102. [000109] In Figures 6 and 7 it is assumed that method 200 is performed by the battery charger 106 and the training method(s) 300(a,b) are performed by the server 102. However, as mentioned above, it may be possible for 300a to be performed by server 102, which then sends an initially trained charging model to the battery charger 106 such as Figure 5, described in more detail below shows, but later for the charging model to self-learn in some embodiments. In other embodiments, the initial pre-training may not take place, and the battery charger 106 only learns what is the best cause of action to follow. In some embodiments, the battery charger 106 may from time to time, receive retrained charging models from server 102 and/or may receive variables to update its charging model. [000110] In the example embodiment of the battery charger apparatus 106 shown in Figure 5, the apparatus comprises at least one or more processors or processing circuitry 402, memory 404 which holds computer code configured when executed to implement a method of charging a battery as disclosed herein. Some embodiments of a computer-implemented method 200 for power management when charging an electric vehicle 100 at a charging power source 106 as disclosed herein may be implemented by executing code in the form, for example, of modules M202, M204, M206, M208, M210, and M212 where each module causes a corresponding method feature S202, S204, S206, S208, S210, S212 as shown in Figure 4A and described hereinabove to be implemented in some embodiments. For example, module S202 may be used to detect a current state of energy of a battery of a vehicle for example such as a battery of a vehicle which may be recharged using a battery charging to implement an embodiment of method 200 as disclosed herein. [000111] In some embodiments, the battery charger 106 is a communications enabled battery charger including a suitable communications module and receiver/transmitter circuitry 406 and may include an antenna to support wireless communications using one or more communications protocols with at least server 106 and soft-switch 104. For example, cellular or Wi-Fi communications protocols may be used to communicate with server 102 via a suitable cellular network or Wi-Fi network access point in some embodiments whereas Wi-Fi or Bluetooth™ communications protocols may be used to communicate with soft-switch 104. As shown in Figure 5, executing the charging model algorithm to implement the method 200 may result in an action to charge or stop charging the electric vehicle at a particular point in time, and this decision may be actuated by a soft-switch controller 408 is some embodiments causing the relay of the soft-switch to be actuated. In some embodiments, the controller 408 may cause a control signal to be sent to the soft-switch using either as a wireless signal or over a wired communications link. In some embodiments, the soft-switch 104 may be provided as a component part within the battery charger 106 however, in some embodiments, the soft-switch 104 may be retrofitted and configured as a separate component. [000112] Figure 6 shows an example embodiment of server apparatus 102 comprising one or more processor(s) or processing circuitry 502, memory 504 which holds computer code 510 which, when executed, causes the apparatus 102 to implement one or more training methods 300, 300a, 300b for configuring a charging model for a battery charger according to any of the disclosed embodiments. As shown in Figure 6, the charging model is configured to be trained using computer code provided in the form of one or more modules M301,M302,M304,M306, M308, and M310, where each module comprise code configured to implement the corresponding S301, S302,S304, S306, S308, and S310 features of the training method shown in Figure 4B and described hereinabove. [000113] Figure 5 shows schematically an example data flow when training a reinforcement learning technology. In Figure 5, the charging

server 102. [000114] This initial training may be advantageous as it helps limit or avoid any inconvenience to a user in the early stages of the battery trained to some extent. [000115]

charger 100 which provides charging power supply to vehicle 100 via a soft-switch 104. Base on the variables and model parameters determined by the training performed, for example, using a method such as method 300 shown in Figure 4B, the server 102 is able configure and reconfigure the charging model which is hosted on the battery charger 106 configured to provide a charging power supply to a connected vehicle 100 whenever the charging model outputs an action to charge the vehicle. Whenever the charging model determines an action to charge an electric vehicle should be taken by the battery charger 106, the battery charger 106 sends an actuation control signal to actuate the soft-switch to allow the vehicle (assuming it is connected) to be charged. In some embodiments, prior to starting charging, a check is performed to confirm if a vehicle is connected or not and an alert may be generated to flag to a user associated with the battery charger and/or vehicle to connect the vehicle for charging. [000116] As shown in Figure 7, the soft switch 104 may be configured to detect when a vehicle connects for subsequent charging and it may be configured to obtain the current state of battery energy. Alternatively, these steps may be performed by the battery charger via the soft- switch and the soft-switch only actuated to provide a power supply when actuated by the battery charger’s soft-switch controller 408. which is actuated by the charging power supply according to the model. [000117] As shown in Figure 7, the soft-switch 104 sends connected vehicle battery information to the battery charger 106 which inputs this into the trained charging model. The charging model then processes the input and outputs an action to charge or not charge the vehicle, or for example, when to start charging the connected vehicle. Once conditions are met to start charging, the battery charger generates a control signal to actuate the soft-switch 104 and once the soft-switch receives the control signal it actuates and allows the vehicle 100 to charge. [000118] In some embodiments, the vehicle may be an autonomous electric vehicle with an ADS configured to make tactical decisions for a control system, for example, it may determine when to dock at the battery charger for recharging. [000119] In some embodiments, the vehicle may be a heavy, also known as heavy-duty, electric vehicle. A heavy-duty vehicle may comprise a wide range of different physical devices, such as combustion engines, electric machines, friction brakes, regenerative brakes, shock absorbers, air bellows, and power steering pumps. These physical devices are commonly known as Motion Support Devices (MSD). The MSDs may be individually controllable, for instance such that friction brakes may be applied at one wheel, i.e., a negative torque, while another wheel on the vehicle, perhaps even on the same wheel axle, is simultaneously used to generate a positive torque by means of an electric machine. The autonomous operation of a heavy-duty vehicle is accordingly more complex than the autonomous operation of a more light-weight vehicle such as a car and the power requirements more complex. Moreover, a heavy-duty vehicle may be electrically connected to a vehicle accessory which may also be charged via the battery charger 106. In some embodiments, the charging model may also monitor separately and/or collectively the state of charge of a connected vehicle accessory to the vehicle connected to the soft-switch and may also charge a battery of the connected vehicle accessory using a separate charging model for that vehicle accessory or using the same charging model as for the vehicle the accessory is electrically coupled to. [000120] Some, if not all, of the above embodiments may be implemented using computer program code which may be provided as software or hardcoded, for example, as a computer program product configured to be used by a device mounted on or integrated in the battery charger 106 or server 102. [000121] For example, the methods 200, 300 described above may be at least partly implemented through one or more processors, such as, the processors or processing circuitry 402, 502 shown in Figures 5 and 6 together with computer program code 410, 510 for performing the functions and actions of the embodiments herein. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code or code means for performing the embodiments herein when being loaded into the processing circuitry in a suitable controller or control unit of the apparatus 106, 104. [000122] The data carrier, or computer readable medium, may be one of an electronic signal, optical signal, radio signal or computer-readable storage medium. [000123] Those skilled in the art will also appreciate that the processing circuitry and the memory or computer readable storage unit described above may refer to a combination of analog and digital circuits, and/or one or more processors configured with software and/or firmware, e.g. stored in a memory, that when executed by the one or more processors such as the processing circuitry perform as described above. One or more of these processors, as well as the other digital hardware, may be included in a single application-specific integrated circuit (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a-chip (SoC). [000124] The controllers 408, 508 may also comprise or be capable of controlling how signals are sent wirelessly via antenna 70 in order for the vehicle 12 to communicate via one or more communications channels with remote entities, for example, a site back office. [000125] In some embodiments of the disclosed technology, an apparatus comprises a server such as server 102 which is configured to communicate soft-switch actuator signals wirelessly to the electric vehicle and/or the battery charger. In some embodiments, when an electric vehicle receives a soft-switch actuator signal, it forwards this to actuate the soft-switch connected to the battery charger. In this way, an electric vehicle may be configured to self-regulate its own charging by forwarding the soft-switch actuator signals to the actuator switch [000126] The communication channels may be point-to-point, or networks, for example, over cellular or satellite networks which support wireless communications. The wireless communications may conform to one or more public or proprietary communications standards, protocols and/or technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol for email (e.g., Internet message access protocol (IMAP) and/or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), and/or Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS)), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document. [000127] The operating systems of the apparatus 106, 108 and of the soft-switch 104 may further various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and for facilitating communication between various hardware and software components. [000128] Where the disclosed technology is described with reference to drawings in the form of block diagrams and/or flowcharts, it is understood that several entities in the drawings, e.g., blocks of the block diagrams, and also combinations of entities in the drawings, can be implemented by computer program instructions, which instructions can be stored in a computer-readable memory, and also loaded onto a computer or other programmable data processing apparatus. Such computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer and/or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, create means for implementing the functions/acts specified in the block diagrams and/or flowchart block or blocks. [000129] In some implementations and according to some aspects of the disclosure, the functions or steps noted in the blocks can occur out of the order noted in the operational illustrations. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality/acts involved. Also, the functions or steps noted in the blocks can according to some aspects of the disclosure be executed continuously in a loop. [000130] The description of the example embodiments provided herein have been presented for the purposes of illustration. The description is not intended to be exhaustive or to limit example embodiments to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired from practice of various alternatives to the provided embodiments. The examples discussed herein were chosen and described in order to explain the principles and the nature of various example embodiments and its practical application to enable one skilled in the art to utilize the example embodiments in various manners and with various modifications as are suited to the particular use contemplated. The features of the embodiments described herein may be combined in all possible combinations of methods, apparatus, modules, systems, and computer program products. It should be appreciated that the example embodiments presented herein may be practiced in any combination with each other. [000131] It should be noted that the word “comprising” does not necessarily exclude the presence of other elements, features, functions, or steps than those listed and the words “a” or “an” preceding an element do not exclude the presence of a plurality of such elements, features, functions, or steps. It should further be noted that any reference signs do not limit the scope of the claims, that the example embodiments may be implemented at least in part by means of both hardware and software, and that several “means”, “units” or “devices” may be represented by the same item of hardware. [000132] The various example embodiments described herein are described in the general context of methods, and may refer to elements, functions, steps or processes, one or more or all of which may be implemented in one aspect by a computer program product, embodied in a computer-readable medium, including computer-executable instructions, such as program code, executed by computers in networked environments. [000133] A computer-readable medium may include removable and non-removable storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory, RAM), which may be static RAM, SRAM, or dynamic RAM, DRAM. ROM may be programmable ROM, PROM, or EPROM, erasable programmable ROM, or electrically erasable programmable ROM, EEPROM. Suitable storage components for memory may be integrated as chips into a printed circuit board or other substrate connected with one or more processors or processing modules, or provided as removable components, for example, by flash memory (also known as USB sticks), compact discs (CDs), digital versatile discs (DVD), and any other suitable forms of memory. Unless not suitable for the application at hand, memory may also be distributed over a various forms of memory and storage components, and may be provided remotely on a server or servers, such as may be provided by a cloud-based storage solution. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes. [000134] The memory used by any apparatus whatever its form of electronic apparatus described herein accordingly comprise any suitable device readable and/or writeable medium, examples of which include, but are not limited to: any form of volatile or non-volatile computer readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and/or any other volatile or non-volatile, non-transitory device readable and/or computer-executable memory devices that store information, data, and/or instructions that may be used by processing circuitry. Memory may store any suitable instructions, data or information, including a computer program, software, an application including one or more of logic, rules, code, tables, etc. and/or other instructions capable of being executed by processing circuitry and, utilized by the apparatus in whatever form of electronic apparatus. Memory may be used to store any calculations made by processing circuitry and/or any data received via a user or communications or other type of data interface. In some embodiments, processing circuitry and memory are integrated. Memory may be also dispersed amongst one or more system or apparatus components. For example, memory may comprises a plurality of different memory modules, including modules located on other network nodes in some embodiments. [000135] In the drawings and specification, there have been disclosed exemplary aspects of the disclosure. However, many variations and modifications can be made to these aspects which fall within the scope of the accompanying claims. Thus, the disclosure should be regarded as illustrative rather than restrictive in terms of supporting the claim scope which is not to be limited to the particular examples of the aspects and embodiments described above. The invention which is exemplified herein by the various aspects and embodiments described above has a scope which is defined by the following claims.

CLAIMS 1. A computer-implemented method (200) for power management when charging an electric vehicle (100) at a charging power source (106), the method comprising: detecting a current state of energy of a battery of a vehicle (S202); determining a target charged state of energy of the battery at a future point in time (S204); and determining a point in time when power should be provided to charge the vehicle from the charging power source based on the detected current state of energy of the battery and the determined target charged state of energy of the battery (S206) so that the future point in time is a predicted end of a charging time period, wherein the future point in time is determined using a vehicle battery charging model configured to determine the target charged battery energy state based on a predicted user demand for power from the charged vehicle at the end of the charging period and at least one charging constraint for the charging power source. 2. The method of claim 1, wherein the method further comprises training the vehicle battery charging model to: map input to the model comprising a set of power load variables of the charging power source to an output representing a charging action to charge the battery or a non-charging action to not charge the battery by rewarding for long-term and short-term consequences of the charging or non-charging actions. 3. The method of claim 2, wherein the method further comprises: when the battery is determined to start charging at the future point in time, based on the determined current state of energy of the battery and a determined charging action of the dynamic vehicle battery charging model, configuring the external charging power supply to automatically provide power to charge the vehicle by actuating a soft switch (S208). 4. The method of either claim 1 or claim 2, wherein based on the predicted future point in time for achieving the determined target charged battery state, the method further comprises, responsive to determining that power should start to be provided to charge the vehicle to achieve the predicted target charged battery state by the determined future time, causing an alert to be provided to a user to connect the vehicle to a power supply if the vehicle is not already connected (S210). 5. The method of any one of the previous claims, wherein at least one charging constraint is a charge or not condition based on an objective to minimize long term energy costs. 6. The method of any one of the previous claims, wherein the set of power load variables of the external power source includes a power load variable of an external power supply to the charging power source, wherein the external power supply is connected to one or more other power-consuming devices consuming power in a charging period. 7. The method of claim 6, wherein the charging constraint comprises a maximum available power constraint for the external power supply to provide power to the charging power supply. 8. The method of claim 6, wherein the power-consuming devices include one or more devices which are not battery operated. 9. The method of any one of the previous claims, wherein the charging model is trained to determine, for each of a plurality of charging periods, electricity cost and a probable risk of a battery state of energy not meeting a target charged battery energy state at the end of that charging period. 10. The method of any one of the previous claims, wherein the method further comprises: varying the amount of charging power used by the charging power supply to charge the vehicle battery based on the current vehicle battery state of energy. 11. The method of any one of the previous claims, wherein the method further comprises: obtaining data generated by implementing the dynamic vehicle battery charging model at the charging power supply; using the gathered data as a training data set for training the dynamic battery vehicle charging model, wherein the gathered data comprises, for each charging period, data representing the start time, the end time, the initial battery charge state, the end battery charge set, the power consumed charging the battery and the cost of the power consumed charging the battery to the charging model (S212), and re-training the trained dynamic battery vehicle charging model by inputting the gathered data into the trained model. 12. The method of any one of the previous claims, wherein the electric vehicle comprises a heavy electric vehicle. 13. The method of any one of the previous claims, wherein the electric vehicle includes an electrically connected vehicle accessory have a battery system capable of being charged via the electric vehicle. 14. The method of any one of the previous claims, wherein the electric vehicle comprises an autonomous, semi-autonomous or remotely controlled electric vehicle. 15. A method of training a charging model using reinforcement learning (300), the charging model being configured for use in a method of any one of claims 1 to 11, the trained model being configured to determine when to charge the vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods, the method comprising: inputting training data to the charging policy model (S302) ; and iteratively processing the input training data to provide iterative output representing a charging cost (S304), wherein each iteration comprises: comparing a charging cost with a target charging cost or with a charging cost from a previous iteration (S306); and maximising cumulative rewards for reducing the charging cost (S308) to achieve the target battery charge state. 16. The method of claim 15, wherein each reward, r, of the maximised cumulative rewards, can be represented for each charging period as either: a charge power consumption per unit time multiplied by a duration of time over which charge power is consumed multiplied by the energy consumption cost per unit of power in the case where the end of the charging period is reached and the target battery energy state is reached; or, if the target battery energy state is not reached at the end of the charging period, a total energy consumption cost for the duration of time the battery was charged plus a penalty term comprising the energy deficiency at the end of the charging period multiplies by a penalty constant. 17. The method of claim 15 or 16, wherein each reward r of the maximised cumulative rewards is determined using a reward function based on a charge power consumption per unit time, a duration of time over which charge power is consumed, a battery energy deficiency comprising a difference between a target state of battery energy and a current state of battery energy, an energy consumption cost, and wherein the reward function indicates if an end of a charging episode is reached or not, and includes a penalty term defining how a violation from a target state of energy is to be penalised. 18. The method of any one of claims 15 to 17, wherein the electric vehicle comprises a heavy electric vehicle. 19. The method of any one of claims 15 to 18, wherein the electric vehicle includes an electrically connected vehicle accessory have a battery system capable of being charged via the electric vehicle. 20. The method of any one of claims 15 to 19, wherein the electric vehicle comprises an autonomous, semi-autonomous or remotely controlled electric vehicle. 21. An apparatus comprising a computer-program product comprising a set of instructions stored in a memory and comprising one or more modules or circuitry which, when loaded from the memory for execution by one or more processors or processing circuitry of the apparatus cause the apparatus to implement a method according to any one of claims 1 to 14 and/or 15 to 20. 22. The apparatus of claim 21, wherein the apparatus is configured to be connected to a grid power supply and comprises a charging power supply outlet configured to be connected to an actuatable a soft-switch 104, wherein, when an electric vehicle 100 is connected to the apparatus, the apparatus is configured to actuate the soft-switch to control a supply of electrical power to at least start charging a battery of the connected electric vehicle based on the apparatus implementing a method according to any one of claims 1 to 15 which determines an action to charge a connected vehicle should be performed. 23. The apparatus of claim 22, wherein actuating the soft-switch 104 also controls the supply of electrical power to stop charging a battery of the connected electric vehicle when the target charged state of energy of the battery is reached. 24. The apparatus of claim 23, wherein the apparatus is configured to actuate the soft-switch 104 to control the start and stop supply of electrical power multiple times whilst the electric vehicle is connected to the apparatus until the target charged state of energy of the battery is reached.

Claims

25. The apparatus of any one of claims 21 to 24, wherein the apparatus comprises a battery charger 106. 26. The apparatus of any one of claims 21 to 24, wherein the apparatus comprises a server 102 connected to a battery charger 106. 27. The apparatus of any one of claims 22 to 24, wherein the electric vehicle comprises a heavy electric vehicle. 28. The method of any one of claims 22 to 25, wherein the electric vehicle includes an electrically connected vehicle accessory have a battery system capable of being charged via the electric vehicle. 29. The method of any one of claims 22 to 26, wherein the electric vehicle comprises an autonomous, semi-autonomous or remotely controlled electric vehicle. 30. The method of any one of claims 22 to 26, wherein the apparatus comprises a server (102) configured to communicate soft-switch actuator signals wirelessly to the electric vehicle, wherein the electric vehicle is configured to self-regulate its own charging by forwarding the soft-switch actuator signals to the actuator switch.

ABSTRACT A computer-implemented method (200) for charging an electric vehicle at a charging power source and related aspects involve detecting a current state of energy of a battery of a vehicle (S202), determining a target charged state of energy of the battery at a future point in time (S204); and determining a point in time when power should be provided to charge the vehicle from the charging power source based on the detected current state of energy of the battery and the determined target charged state of energy of the battery (S206) so that the future point in time is a predicted end of a charging time period, wherein the future point in time is determined using a dynamic vehicle battery charging model configured to determine the target charged battery energy state based on a predicted user demand for power from the charged vehicle at the end of the charging period and at least one charging constraint for the charging power source. To accompanying the published abstract: Figures 3A and 3B

12. The method of any one of the previous claims, wherein the electric vehicle comprises a heavy electric vehicle.

13. The method of any one of the previous claims, wherein the electric vehicle includes an electrically connected vehicle accessory have a battery system capable of being charged via the electric vehicle.

14. The method of any one of the previous claims, wherein the electric vehicle comprises an autonomous, semi-autonomous or remotely controlled electric vehicle.

15. A method of training a charging model using reinforcement learning (300), the charging model being configured for use in a method of any one of claims 1 to 11, the trained model being configured to determine when to charge the vehicle based on predicted charging costs for an end state of battery energy to meet a predicted target charged battery energy state at the end of the one or more charging periods, the method comprising: inputting training data to the charging policy model (S302) ; and iteratively processing the input training data to provide iterative output representing a charging cost (S304), wherein each iteration comprises: comparing a charging cost with a target charging cost or with a charging cost from a previous iteration (S306); and maximising cumulative rewards for reducing the charging cost (S308) to achieve the target battery charge state.

16. The method of claim 15, wherein each reward, r, of the maximised cumulative rewards, can be represented for each charging period as either: a charge power consumption per unit time multiplied by a duration of time over which charge power is consumed multiplied by the energy consumption cost per unit of power in the case where the end of the charging period is reached and the target battery energy state is reached; or, if the target battery energy state is not reached at the end of the charging period, a total energy consumption cost for the duration of time the battery was charged plus a penalty term comprising the energy deficiency at the end of the charging period multiplies by a penalty constant.

17. The method of claim 15 or 16, wherein each reward r of the maximised cumulative rewards is determined using a reward function based on a charge power consumption per unit time, a duration of time over which charge power is consumed, a battery energy deficiency comprising a difference between a target state of battery energy and a current state of battery energy, an energy consumption cost, and wherein the reward function indicates if an end of a charging episode is reached or not, and includes a penalty term defining how a violation from a target state of energy is to be penalised.

18. The method of any one of claims 15 to 17, wherein the electric vehicle comprises a heavy electric vehicle.

19. The method of any one of claims 15 to 18, wherein the electric vehicle includes an electrically connected vehicle accessory have a battery system capable of being charged via the electric vehicle.

20. The method of any one of claims 15 to 19, wherein the electric vehicle comprises an autonomous, semi-autonomous or remotely controlled electric vehicle.

21. An apparatus comprising a computer-program product comprising a set of instructions stored in a memory and comprising one or more modules or circuitry which, when loaded from the memory for execution by one or more processors or processing circuitry of the apparatus cause the apparatus to implement a method according to any one of claims 1 to 14 and/or 15 to 20.

22. The apparatus of claim 21, wherein the apparatus is configured to be connected to a grid power supply and comprises a charging power supply outlet configured to be connected to an actuatable a soft-switch 104, wherein, when an electric vehicle 100 is connected to the apparatus, the apparatus is configured to actuate the soft-switch to control a supply of electrical power to at least start charging a battery of the connected electric vehicle based on the apparatus implementing a method according to any one of claims 1 to 15 which determines an action to charge a connected vehicle should be performed.

23. The apparatus of claim 22, wherein actuating the soft-switch 104 also controls the supply of electrical power to stop charging a battery of the connected electric vehicle when the target charged state of energy of the battery is reached.

24. The apparatus of claim 23, wherein the apparatus is configured to actuate the soft-switch 104 to control the start and stop supply of electrical power multiple times whilst the electric vehicle is connected to the apparatus until the target charged state of energy of the battery is reached.

25. The apparatus of any one of claims 21 to 24, wherein the apparatus comprises a battery charger 106.

26. The apparatus of any one of claims 21 to 24, wherein the apparatus comprises a server 102 connected to a battery charger 106.

27. The apparatus of any one of claims 22 to 24, wherein the electric vehicle comprises a heavy electric vehicle.

28. The method of any one of claims 22 to 25, wherein the electric vehicle includes an electrically connected vehicle accessory have a battery system capable of being charged via the electric vehicle.

29. The method of any one of claims 22 to 26, wherein the electric vehicle comprises an autonomous, semi-autonomous or remotely controlled electric vehicle.

30. The method of any one of claims 22 to 26, wherein the apparatus comprises a server (102) configured to communicate soft-switch actuator signals wirelessly to the electric vehicle, wherein the electric vehicle is configured to self-regulate its own charging by forwarding the soft-switch actuator signals to the actuator switch.