KR20230119883A - Operation Method for Power of energy storage system and Power Management Device using the same - Google Patents

Abstract

The present invention includes the steps of collecting Home data of an energy storage system related to the load of an area where at least one electric vehicle is located and EV data according to the use of the electric vehicle, storing the EV data and the Home data, and the stored EV. Based on data and the Home data, reinforcement learning is performed so that the state of charge of the battery of the electric vehicle exceeds a predefined user satisfaction level while the state of charge of the energy storage system maintains a predefined optimal charge amount corresponding to the load. Disclosed is a power management method of an energy storage system including the step of establishing a policy accordingly, and a power management device supporting the same.

Description

ESS power operation method and power management device supporting the same {Operation Method for Power of energy storage system and Power Management Device using the same}

The present invention relates to a SoC of an ESS, and more particularly, to a power management method for SoC management of an ESS in consideration of user satisfaction and a power management device supporting the same.

ESS (Energy Storage System, ESS) is a method that can be used for peak shaving, load shifting, etc. through surplus power storage. Such an ESS can be utilized like an uninterruptible power supply (UPS) to improve the quality of power. The State of Charge (SoC) of ESS must have sufficient values to cope with peaks that may occur at any time. In addition, the optimal SoC must be maintained due to factors such as battery life and electric bill. However, since SoC of ESS is determined by power demand and supply, its value has passive characteristics. In addition, the SoC of the ESS has a limited capacity, so if power is supplied in a state in which the SoC is exceeded, the remaining power is wasted. Therefore, SoC of ESS needs to be optimized.

On the other hand, Demand Response (DR) is one of the methods for efficiently consuming power through interoperability between power providers and consumers. The DR power management method can be used for power management, such as reducing peak demand or securing reserve power. DR power management methods include Interruptible Load (IL), which reduces user demand power, and Vehicle to Grid (V2G), which supplies energy from vehicles to the grid. Here, when the DR power management method is performed, there is a concern that user satisfaction may decrease. In the case of IL, since the power supplied to the user is reduced, the quality of service provided to the user may be degraded. Since V2G also consumes the SoC of an Electric Vehicle (EV) to cope with peak demand, a problem may occur that the SoC of the EV that the user needs at the time of using the EV is not satisfied.

Accordingly, the present invention provides a power management method of an ESS capable of securing user satisfaction in applying a DR power management method using reinforcement learning and a power management device supporting the same.

In addition, the present invention provides an ESS power management method capable of achieving optimization of the ESS SoC and minimization of deterioration in usage satisfaction when calculating a reward, and a power management device supporting the same.

However, the object of the present invention is not limited to the above object, and other objects not mentioned will be clearly understood from the description below.

In order to achieve the above object, a power management method of an energy storage system includes controlling charging of an energy storage device supplying power to a load and a battery of at least one electric vehicle to a region where the at least one electric vehicle is located. Collecting Home data of the energy storage system related to the load of the electric vehicle and EV data according to the use of the electric vehicle, storing the EV data and the Home data, and storing the energy based on the stored EV data and the Home data. Performing reinforcement learning so that the state of charge of the battery of the electric vehicle exceeds a predefined user satisfaction level while maintaining the state of charge of the system at a predefined optimal charge amount corresponding to the load, and establishing a policy accordingly. It is characterized by doing.

The method further comprises generating an ESS control signal for controlling the state of charge of the energy storage system and a vehicle control signal for controlling the state of charge of the battery of the at least one electric vehicle according to the establishment of the policy. to be

In addition, the method includes supplying the ESS control signal to the energy storage system to control the amount of charge of the energy storage system, and supplying the vehicle control signal to the at least one electric vehicle to control the battery of the at least one electric vehicle. It is characterized in that it further comprises the step of controlling the filling amount.

In this case, the step of establishing the policy is to establish a policy that controls the charging amount differently according to the usage time and usage distance of each electric vehicle while maintaining the charged amount of the energy storage system to correspond to the load through the reinforcement learning. It is characterized by including steps.

In addition, the step of establishing a policy for controlling the amount of charge differently according to the use time and the use distance may include relatively high and fast charging as the use time of the electric vehicle approaches and as the use distance of the electric vehicle increases through the reinforcement learning. Establishing a policy for controlling the charging to be performed or establishing a policy for controlling such that relatively low and slow charging is performed as the usage time of the electric vehicle is far and the usage distance of the electric vehicle is short through the reinforcement learning. It is characterized by doing.

Meanwhile, the step of establishing the policy sets the battery charging speed and basic charging amount of the at least one electric vehicle relatively high in a section in which surplus power of the energy storage system is generated through the reinforcement learning, and the energy storage system and establishing a policy for setting a battery charging rate and a basic charging amount of the at least one electric vehicle relatively low in a period in which a peak demand occurs.

A power management device for supporting power management of an energy storage system according to an embodiment of the present invention includes Home data on the state of charge of the energy storage system related to the load of an area where at least one electric vehicle is located and EV according to the use of the electric vehicle. A communication circuit for receiving data, a memory for storing the Home data and the EV data, and a processor functionally connected to the communication circuit and the memory, wherein the processor performs the communication based on the stored EV data and the Home data. It is set to perform reinforcement learning so that the SOC of the electric vehicle battery SOC of the electric vehicle exceeds a predefined user satisfaction level while maintaining the SOC of the energy storage system at a predefined optimal amount corresponding to the load, and establishes a policy accordingly. characterized by

Here, the processor may establish a policy to have a different charge amount according to the use time and use distance of each electric vehicle while maintaining the charge amount of the energy storage system to correspond to the load through the reinforcement learning.

In addition, the processor establishes a policy to control relatively high and fast charging as the usage time of the electric vehicle approaches and as the usage distance of the electric vehicle increases through the reinforcement learning, or through the reinforcement learning, the A policy for controlling a relatively low and slow charging operation may be established as the use time of the electric vehicle is longer and the use distance of the electric vehicle is shorter.

In addition, the processor sets the battery charging rate and basic charging amount of the at least one electric vehicle relatively high in a section in which surplus power of the energy storage system is generated through the reinforcement learning, and the peak demand of the energy storage system In a section where ? occurs, a policy of setting the battery charging rate and basic charging amount of the at least one electric vehicle relatively low may be established.

According to the present invention, it is possible to prevent user damage by applying user feedback to reinforcement learning to achieve optimization of the ESS SoC and maintain user satisfaction above a specified standard.

In addition, various effects other than the above effects may be disclosed directly or implicitly in detailed descriptions according to embodiments of the present invention to be described later.

1 is a diagram illustrating an example of an ESS power management environment supporting an ESS power management method according to an embodiment of the present invention.
2 is a diagram showing an example of a configuration of an electric vehicle according to an embodiment of the present invention.
3 is a diagram illustrating an example of a configuration of a power management device for managing an energy storage system according to an embodiment of the present invention.
4 is a diagram illustrating an example of an ESS power management method according to an embodiment of the present invention.
5 is a diagram illustrating another example of a processor configuration among configurations of an ESS power management apparatus supporting ESS power management according to an embodiment of the present invention.
6 is a diagram showing a configuration of a data acquisition unit according to an embodiment of the present invention in more detail.
7 is a diagram showing the configuration of an EV data management unit according to an embodiment of the present invention in more detail.
8 is a diagram showing the configuration of a Home data management unit according to an embodiment of the present invention in more detail.
9 is a diagram showing a step configuration according to an embodiment of the present invention in more detail.
10 is a diagram showing an agent configuration in more detail according to an embodiment of the present invention.

In order to clarify the characteristics and advantages of the problem solving means of the present invention, the present invention will be described in more detail with reference to specific embodiments of the present invention shown in the accompanying drawings.

However, detailed descriptions of well-known functions or configurations that may obscure the gist of the present invention will be omitted in the following description and accompanying drawings. In addition, it should be noted that the same components are indicated by the same reference numerals throughout the drawings as much as possible.

The terms or words used in the following description and drawings should not be construed as being limited to a common or dictionary meaning, and the inventor may appropriately define the concept of terms for explaining his/her invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that there is. Therefore, the embodiments described in this specification and the configurations shown in the drawings are only one of the most preferred embodiments of the present invention, and do not represent all of the technical ideas of the present invention. It should be understood that there may be equivalents and variations.

In addition, terms including ordinal numbers, such as first and second, are used to describe various components, and are used only for the purpose of distinguishing one component from other components, and to limit the components. Not used. For example, a second element may be termed a first element, and similarly, a first element may be termed a second element, without departing from the scope of the present invention.

In addition, terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In addition, terms such as "include" or "having" described in this specification are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or the other It should be understood that the above does not preclude the possibility of the presence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof.

In addition, terms such as “unit”, “unit”, and “module” described in the specification refer to a unit that processes at least one function or operation, and may be implemented as hardware or software or a combination of hardware and software. Also, "a or an", "one", "the" and similar related words in the context of describing the invention (particularly in the context of the claims below) Unless indicated or otherwise clearly contradicted by context, both the singular and the plural can be used.

In addition to the above-mentioned terms, specific terms used in the following description are provided to aid understanding of the present invention, and the use of these specific terms may be changed in other forms without departing from the technical spirit of the present invention.

In addition, embodiments within the scope of the present invention include computer-readable media having or conveying computer-executable instructions or data structures stored thereon. Such computer readable media can be any available media that can be accessed by a general purpose or special purpose computer system. By way of example, such computer readable media may be in the form of RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, or computer executable instructions, computer readable instructions or data structures. physical storage media such as, but not limited to, any other medium that can be used to store or convey any program code means in a computer system and which can be accessed by a general purpose or special purpose computer system. .

1 is a diagram showing an example of an ESS power management environment supporting an ESS power management method according to an embodiment of the present invention, and FIG. 2 is a diagram showing an example of an electric vehicle configuration according to an embodiment of the present invention, 3 is a diagram illustrating an example of a configuration of a power management device for managing an energy storage system according to an embodiment of the present invention.

Referring to FIG. 1 , the power management environment 10 of the energy storage system of the present invention includes a power management device 100 that manages the energy storage system, a power generation source 20, an energy storage device 110, and a load 30. and at least one electric vehicle 200.

The power generation source 20 may include at least one generator that supplies power to the power management device 100 . The power generation source 20 may be at least one of various types of power generation such as thermal power generation, hydroelectric power generation, solar power generation, wind power generation, and nuclear power generation, and thus the present invention is not limited to the type of power generation source 20. , may mean a subject that generates power through at least one power generation so as to supply power to the power management device 100 .

The load 30 may be a consuming entity that receives power generated by the power generation source 20 through the power management device 100 or the energy storage device 110 . The load 30 may include various types of sources capable of consuming the supplied power, such as a general household, a factory, or a store. Here, the present invention is not limited by the type of the load 30 or the amount of power consumed by the load 30 .

The energy storage device 110 may include a battery device installed at a designated point capable of supplying power to at least one electric vehicle 200 . In addition, the energy storage device 110 may be installed adjacent to the load 30 capable of supplying power to the load 30 . It has been described that the above-described energy storage device 110 can be disposed adjacent to the load 30 so as to supply power to subjects (eg, the load 30) that store power, but the present invention is limited thereto. it is not going to be For example, the energy storage device 110 may be installed at a place separated from the load 30 by a predetermined distance or more and supply power to the load 30 through a cable or the like. Also, the energy storage device 110 may be disposed close to the power source 20 . Alternatively, the energy storage device 110 may be disposed together with the power management device 100 .

The at least one electric vehicle 200 (hereinafter referred to as an electric vehicle) may receive and store power from the energy storage device 110 . In addition, the electric vehicle 200 may provide stored electricity to the energy storage device 110 according to the request of the power management device 100 . In this regard, as shown in FIG. 2 , the electric vehicle 200 includes a vehicle communication circuit 210, a sensor unit 220, a vehicle memory 230, a vehicle display 240, an input device 260, and movement. It may include a means 270 , a battery 280 and a vehicle processor 250 configuration.

The vehicle communication circuit 210 may form a communication channel with the power management device 100 . The vehicle communication circuit 210 may provide information (eg, EV data) related to the electric vehicle 200 to the power management device 100 in response to control of the vehicle processor 250 . For example, the vehicle communication circuit 210 provides information on the remaining amount of the battery 280 of the electric vehicle 200, the current location of the electric vehicle 200, a movement history of the electric vehicle 200, and a user input related to the operation of the electric vehicle 200. Information and the like may be provided to the power management device 100 . When the energy storage device 110 includes the communication circuit, the vehicle communication circuit 210 forms a communication channel with the power management device 100 through the energy storage device 110 or the power management device through the base station. (100) and can form a communication channel. Alternatively, the electric vehicle 200 may form a communication channel with the power management device 100 through a cable connected in the process of charging the battery 280 through the energy storage device 110 . Accordingly, the vehicle communication circuit 210 is not limited to a specific communication method or communication type, and at least one communication circuit (eg, wired or wireless communication circuit).

The sensor unit 220 may include at least one sensor capable of collecting various information sensed in relation to the operation of the electric vehicle 200 . For example, the sensor unit 220 may include the current location of the electric vehicle 200, the moving distance of the electric vehicle 200, the moving speed of the electric vehicle 200, the temperature and humidity of the outside and inside of the electric vehicle 200, and the electric vehicle 200. (200) A sensor capable of sensing ambient brightness information and the like may be included. The sensor unit 220 may temporarily or semi-permanently store sensing information collected in response to control of the vehicle processor 250 in the vehicle memory 230 .

The vehicle memory 230 may store at least one program or data related to the operation of the electric vehicle 200 . For example, the vehicle memory 230 may store a sensor operating program for operating the sensor unit 220 and a program for operating the vehicle communication circuit 210 . In addition, the vehicle memory 230 of the present invention collects data (eg, EV data) necessary for reinforcement learning of the power management device 100, and provides the collected data to the power management device 100. Data collected according to program operation can be saved.

The vehicle display 240 may provide various screens related to the operation of the electric vehicle 200 . For example, the vehicle display 240 may display the current operating state of the electric vehicle 200, map information related to the current location of the electric vehicle 200, remaining amount information of the battery 280 of the electric vehicle 200, and electric vehicle 200. In relation to operation, schedule information input by a user, usage information of the power management device 100 for the electric vehicle 200, and the like may be output. When connected to the energy storage device 110 or the power management device 100, the vehicle display 240 may display the charging completion time of the battery 280. In this process, the vehicle ( 200) When receiving an input of the scheduled departure time, the vehicle display 240 may output related information.

The input device 260 may include at least one means capable of receiving a user input related to the use of the electric vehicle 200 . For example, the input device 260 may include a touch pad, a touch screen, a physical key button, a voice input device, and the like. The input device 260 may input a destination and scheduled departure time in response to user manipulation. Such input information may be provided to the power management device 100 through the vehicle communication circuit 210 in response to control of the vehicle processor 250 .

The moving unit 270 may include various devices necessary for moving the electric vehicle 200 . For example, the moving unit 270 may include a plurality of wheels for moving the electric vehicle 200, a shaft connected to the wheels, a gear connected to the shaft, and a gear and a power generating unit. The power generating unit may include, for example, at least one motor, and may be connected to the battery 280 and driven using power supplied by the battery 280 . Additionally, the moving means 270 may include, for example, a steering means for adjusting the steering of wheels, an accelerator and a brake for adjusting speed.

The battery 280 may be disposed at a predetermined location of the electric vehicle 200 to store power supplied from the energy storage device 110 . In addition, the battery 280 transfers stored power in response to the control of the vehicle processor 250 to the moving means 270 and other components (eg, the vehicle display 240, the vehicle communication circuit 210, the sensor unit 220, vehicle memory 230, vehicle processor 250, etc.).

The vehicle processor 250 may transmit and process various signals related to the use of the electric vehicle 200 . For example, if the vehicle processor 250 recognizes a user's approach or a user's boarding through the sensor unit 220, it may output a screen for inputting a destination to be moved through the vehicle display 240. Alternatively, the vehicle processor 250 may control the movement of the electric vehicle 200 according to the user's manipulation of the movement means 270 when there is no separate destination input. In particular, the vehicle processor 250 of the present invention receives information about the scheduled departure time of the vehicle from the user through the input device 260 or stores the user's use history of the electric vehicle 200, and based on this, the vehicle processor 250 receives information about the expected departure time of the vehicle. (200) Departure time can be detected. The vehicle processor 250 provides information about the vehicle use start time to the power management device 100, and requests the power management device 100 to use the charging speed of the battery 280 and the power stored in the battery 280. can be provided according to

The power management device 100 may manage the state of charge of the energy storage device 110 and the state of charge of the electric vehicle 200 . Through this, the power management device 100 efficiently handles the peak demand response of the load 30 and processing of surplus power, and manages the available state of the electric vehicle 200 so that the standby of the user's electric vehicle 200 Satisfaction with the condition can be maintained or improved above a predefined minimum. For example, the power management device 100 adjusts the charging power of the battery 280 of the electric vehicle 200 in consideration of the user's use history of the electric vehicle 200, and adjusts the charging power of the battery 280 as much as possible. The generated surplus power or surplus power can be used to manage the energy storage device 110, and surplus power or surplus power generated by managing the state of charge of the energy storage device 110 can be used to determine the state of charge of the electric vehicle 200. By applying to , it is possible to improve the user's satisfaction with the electric vehicle 200 by adjusting the standby state of the electric vehicle 200 . To this end, the power management device 100 may include a communication circuit 110 , a memory 130 , a display 140 and a processor 150 as shown in FIG. 3 .

The communication circuit 110 is a component necessary for operating the communication channel of the power management device 100, and is not limited to its kind and type. That is, the communication circuit 110 may include a wired or wireless communication circuit capable of forming a communication channel with the energy storage device 110 connected to the load 30, and may also include the electric vehicle 200 or the electric vehicle 200 ) may include various communication circuits capable of communicating with the connected energy storage device 110 . The communication circuit 110 receives Home data including the amount of remaining power of the energy storage device 110 connected to the load 30 or peak demand information or peak demand prediction information of the load 30, and transmits it to the processor 150. can supply to In addition, the communication circuit 110 transmits the connection state of the energy storage device 110 of the electric vehicle 200, the charging state of the electric vehicle 200 or the battery 280 of the electric vehicle 200 to a separate energy storage device ( 110), Home data including information that can be used may be collected and provided to the processor 150. Also, the communication circuit 110 may receive EV data related to the operation of the electric vehicle 200 and provide the received EV data to the processor 150 .

The memory 130 may store various programs and data necessary for operating the power management device 100 . In particular, the memory 130 may store a reinforcement learning algorithm and learned data for managing the ESS SoC and managing the operation of the electric vehicle 200 according to the present invention. In this regard, the memory 130 may store EV data and Home data for reinforcement learning. The learning data stored in the memory 130 may be updated according to a user use history of the electric vehicle 200 or a change in power utilization rate of the load 30 .

The display 140 may output at least one screen related to the operation of the power management device 100 . For example, the display 140 displays a change in the amount of power supplied by the energy storage device 110 to the load 30 and a change in the amount of power supplied or received according to the use of the energy storage device 110 of the electric vehicle 200. can output In addition, the display 140 may receive and output, for each electric vehicle 200, satisfaction according to the state of charge of the battery 280 of the electric vehicle 200 input by users of the electric vehicle 200 . Meanwhile, the screen display of the display 140 is to provide information to the manager who operates the power management device 100, and if the manager does not request a separate display of information, the display 140 may be omitted. can

The processor 150 generates an ESS control signal for controlling the ESS SoC and a vehicle control signal for controlling charging and discharging of the electric vehicle 200, and transmits the generated control signal to a corresponding component, for example, the energy storage device 110 and the electric vehicle. It can be delivered to the vehicle 200. In particular, the processor 150 of the present invention uses the electric vehicle 200 to improve the user satisfaction level of the electric vehicle 200 to a predefined minimum value or higher or to a previous level of satisfaction value while achieving ESS SoC optimization. It is possible to calculate learning data that can improve user satisfaction while maintaining ESS SoC optimization by collecting EV data related to and performing reinforcement learning based on the collected EV data and Home data related to the operation of the energy storage device 110. can In addition, the processor 150 newly generates an ESS control signal and a vehicle control signal based on the calculated learning data, supplies them to corresponding components (eg, the energy storage device 110 and the electric vehicle 200) to operate, Reinforcement learning may be performed by collecting data accordingly (eg, EV data and Home data corresponding to user input according to control signal application) again. Ultimately, the processor 150 calculates a value for a state of the electric vehicle 200 in which user satisfaction with the electric vehicle 200 maintains a specified value or more while maintaining ESS SoC optimization, and for the electric vehicle 200 The vehicle control signal and the ESS control signal may be maintained and managed based on received information about the user's satisfaction or dissatisfaction by providing the operating state to the user (eg, output through the electric vehicle 200). Here, since user satisfaction may vary by environment, location, and user, vehicle control signals may be generated in various ways based on various data, and reinforcement learning is performed to maximize user satisfaction while maintaining ESS SoC optimization. this can be done

As described above, in relation to the DR power management method, the ESS power management environment 10 of the present invention achieves charging (or full charge) at a certain level or higher until the EV starts and home data such as ESS SoC optimization and IL minimization Reinforcement learning is performed based on EV data corresponding to the user satisfaction response through .

4 is a diagram illustrating an example of an ESS power management method according to an embodiment of the present invention.

Referring to FIG. 4 , in relation to the ESS power management method of the present invention, the processor 150 of the power management device 100 may collect EV data and Home data in step 501 . For example, the processor 150 may receive location information of the electric vehicle 200 from the electric vehicle 200 in relation to EV data collection, and obtain weather information corresponding to the location from a weather server. In addition, the processor 150 may form a communication channel with the electric vehicle 200 to collect a user usage history (eg, usage time and distance of the electric vehicle 200) of the electric vehicle 200. . In addition, the processor 150 of the power management device 100 may collect information about the operating state of the energy storage device 110 in the area where the electric vehicle 200 is located in relation to home data collection.

In step 503, the processor 150 of the power management device 100 may store and manage the collected data. The processor 150 may store the collected external data and user data in the memory 130 . In this process, the processor 150 may match and store the collected data with the electric vehicle 200 identifier information. In addition, the processor 150 may provide condition information on the use of the battery 280 with respect to the electric vehicle 200 (eg, a condition allowing the power management device 100 to use the power storage space of the battery 280). , the size of the storage space and the time period of use of the storage space) can be matched and stored. Also, the processor 150 may store information about the electric vehicle 200 and collected data by region, and additionally, the processor 150 may match and store load 30 information by region.

In step 505, the processor 150 of the power management device 100 may perform reinforcement learning for user satisfaction and ESS SoC management. That is, the processor 150 checks ESS SoC information for supplying power to the load 30 in the area where the electric vehicle 200 is located, and based on this, enhances the charging control of the battery 280 of the electric vehicle 200. learning can be done. For example, the power management device 100 may calculate a surplus power generating period or a peak demand generating period in an area where the corresponding load 30 is located through ESS SoC information. In this process, the power management device 100 allocates higher power (or power greater than a predefined reference value) to the charging of the battery 280 of the electric vehicle 200 in the surplus power generation period, or a higher speed (or It can be set to ensure charging at a rate higher than a defined reference rate. In addition, the power management device 100 allocates relatively low power (or power smaller than a predefined reference value) to charging the battery 280 of the electric vehicle 200 in a peak demand period, or allocates a low speed (or predefined power). Control for charging (speed slower than the defined reference speed) can be performed. Additionally, the power management device 100 prioritizes charging of the electric vehicle 200 to be used first, selectively, in consideration of the individual use distance and use time of the electric vehicle 200, and A charging amount for the electric vehicle 200 having a usage distance may be set to be large. As described above, the power management device 100 may control the ESS SoC state management and the charge amount management of the electric vehicle 200 according to complex factors. In the process of performing the above-described operation, the processor 150 may control to establish a policy by performing reinforcement learning to perform an action for controlling the state of charge of the electric vehicle 200 while optimizing the ESS SoC.

Additionally, the processor 150 may store training data through reinforcement learning in the memory 130 or update training data pre-stored in the memory 130 . In addition, the processor 150 generates an ESS control signal and a vehicle control signal corresponding to the learned data, and provides the generated control signal to at least one of the energy storage device 110 and the electric vehicle 200 to obtain the learned It can be controlled to perform power charge/discharge control according to data.

Through the above-described ESS power management method, the power management device 100 can efficiently manage the ESS SoC and improve user satisfaction with the electric vehicle 200 . For example, the power management device 100 determines the basic charging amount of the battery 280, the charging speed of the battery 280, and the battery ( While controlling at least one of the charging times of the battery 280, the power required for charging the battery 280 or the power stored in the battery 280 of the electric vehicle 200 is supplied to the charge of the energy storage device 110 or to the load 30. available. Alternatively, the power management device 100 determines the electric vehicle 200 according to the power supply situation of the energy storage device 110 for each region (eg, the amount of power supplied from the power generation source 20 and the amount of power used by the load 30). User satisfaction by adjusting the basic charging amount of the battery 280 and using the surplus or spare power of the energy storage device 110 to charge the battery 280 of the electric vehicle 200 so that the electric vehicle 200 can be used at any time can increase In this process, the power management device 100 collects information about the user's satisfaction by using the situation or whether the use of the battery 280 is approved and the conditions for using the battery 280 as reinforcement learning data, It is possible to control the ESS SoC to operate efficiently while maintaining a predefined minimum value for user satisfaction.

5 is a diagram showing another example of a processor configuration among configurations of an ESS power management device supporting ESS power management according to an embodiment of the present invention, and FIG. 6 is a diagram showing a data acquisition unit configuration according to an embodiment of the present invention in more detail. is the drawing shown. 7 is a diagram showing the configuration of an EV data management unit in more detail according to an embodiment of the present invention, and FIG. 8 is a diagram showing the configuration of a Home data management unit according to an embodiment of the present invention in more detail. 9 is a diagram showing the configuration of Steps according to an embodiment of the present invention in more detail, and FIG. 10 is a diagram showing the configuration of an agent according to an embodiment of the present invention in more detail.

In FIG. 5 , Environment 153 is composed of data acquisition unit 153a, EV data management unit 153b, Home data management unit 153c, and Step 153d. Agent 154 is divided into an initialization unit 154a, a test agent 154c, and a learning agent 154b. Agent 154 receives State, Reward, and Done from Environment 153, performs learning, and then selects an action. The selected Action is transmitted to the Environment (153). The data acquisition unit 153a of the Environment 153 collects EV-related data and Home-related data. The EV data management unit 153b receives EV-related data from the data acquisition unit 153a and manages it, and the Home data management unit 153c receives Home-related data from the data acquisition unit 153a and manages it. Step (153d) advances Step (153d) of Environment (153) through managed data. In addition, the Step (153d) communicates with the interface (154d) of the Agent (154) to transmit State, Reward, and Done, and receives Action. The Agent 154 initializes each Agent 154 through the initialization unit 154a. The Learning Agent 154b is an Agent for learning and the Test Agent 154c is an Agent for testing the learned result. Each agent 154 can receive State, Reward, and Done by communicating with the interface 154d, and selects an action according to the received information and delivers it to the interface 154d. The interface (154d) communicates with the Step (153d) of the Environment (153) to receive State, Reward, and Done, transfers them to each Agent (154), and delivers the received Action to the Step (153d).

Referring to FIG. 6 , in the data acquisition unit 153a, the EV data reception unit 153a1 receives EV data, converts it into a format suitable for the system, and transmits it to the EV data transmission unit 153a2. The EV data transmitter 153a2 receives EV data from the EV data receiver 153a1 and transmits it to the EV data manager 153b.

The Home data receiver 153a3 similarly receives Home data, converts it into a format suitable for the system, and transmits it to the Home data transmitter 153a4. The home data transmission unit 153a4 receives home data from the home data reception unit 153a3 and transmits it to the home data management unit 153c.

Referring to FIG. 7 , the EV data management unit 153b manages EV-related data. In an embodiment of the present invention, data such as EV ID 153b1, SoC 153b2, arrival time 153b3, departure time 153b4, and requested power amount 153b5 are managed.

Referring to FIG. 8 , the Home data management unit 153c manages Home-related data. The managed data is ESS SoC (153c1), PV (153c2) (power generator using solar energy as a solar generator), NIL (153c3) (a device that cannot interrupt the supply of power, e.g. a computer), IL (153c4) (A device that can temporarily stop supplying power, such as an air conditioner) manages the same data.

Referring to FIG. 9, Step (153d) includes Get Action (153d1) to obtain Action, State Update (153d2) to update State, Calculate Reward (153d3) to calculate Reward, and Calculate Done (153d4) to calculate Done. consists of the functions of First, after receiving an Action from the Agent 154 and receiving current data from the EV data management unit 153b and the Home data management unit 153c, the State is updated through the State Update 153d2. Finally, Step (153d) calculates Reward and Done and retransmits the result to Agent (154).

Referring to FIG. 10, the Agent 154 is divided into an Agent initialization unit 154a, a Test Agent 154c, and a Learning Agent 154b. The agent initialization unit 154a initializes models necessary for learning, such as the MLP Model 154a1 and the RL Agent 154a3, and initializes the Observation Space 154a2 and Action Space 154a4. The test agent (154c) is an agent for experimentation and uses the greedy algorithm (154c_1) to explore the learned results. The learning agent (154b) is an agent for learning and adopts the Epsilon Greedy (154b_1) method for learning.

According to one embodiment, the environment 153 may collect information about a usage time zone and a usage distance of each electric vehicle 200 . At this time, the environment 153 may calculate the usage time zone of the electric vehicle 200 according to EV data, for example, a weather element or a temperature element, and collect usage information of the electric vehicle 200 under external data conditions. The agent 154 generates an individual vehicle control signal based on the usage information of each electric vehicle 200 and transmits it to each electric vehicle 200 to determine the state of charge and charging time of the electric vehicle 200. Reinforcement learning can be performed by controlling to adjust and collecting Home data related to user satisfaction. As a result of reinforcement learning, the agent 154 uses the power stored in the battery 280 of the electric vehicle 200, which is not the time of use of the vehicle, to charge another electric vehicle 200 or supply energy to the load 30. It can be controlled to be used in the storage device 110 .

In addition, the agent 154 of the power management device 100 supplies power for charging the battery 280 from another electric vehicle 200 or the energy storage device 110 in the case of an electric vehicle 200 having a close use time zone. It can be controlled to receive and recharge. In this regard, the environment 153 collects EV data including charging state information of the electric vehicles 200 and Home data including the charging state of the energy storage device 110 to perform reinforcement learning, Control may be performed to adjust the state of charge of the electric vehicle 200 so that the SoC of the energy storage device 110 has a time-optimized value and the user's satisfaction is high.

In addition, in the power management device 100, the Agent 154 may set the amount of charge of the battery 280 to be charged in each electric vehicle 200 in consideration of the usage distance of the individual electric vehicle 200. . For example, the environment 153 of the power management device 100 is a first basic battery 280 of an electric vehicle having a history of traveling a relatively long distance (or a usage distance greater than a predetermined standard usage distance). Second basic charge amount information of the battery 280 of the electric vehicle having a charge amount information and a history of driving a relatively short distance (or a use distance smaller than a use distance of a predetermined standard) may be collected. The Agent 154 conducts reinforcement learning that can increase user satisfaction of each electric vehicle while maintaining the energy storage device in the area where the electric vehicle 200 is located in a state capable of supplying the amount of power required for the corresponding time period. can be done For example, as a result of reinforcement learning, the Agent 154 may set the first basic charge amount information to be greater than the second basic charge amount information. In this process, the Agent 154 of the power management device 100 has a short use distance and use time (eg, a use distance smaller than a predefined reference use distance and the amount of time to elapse between the current time and the use time). A reinforcement learning result may be applied so that the power stored in the battery 280 of the electric vehicle 200, where is shorter than a predefined time, is used to cope with the peak demand of the load 30 as reserve power.

In the above process, the power management device 100 optimizes the ESS SoC and achieves user satisfaction according to the amount of use of the battery 280 of the electric vehicle 200 (eg, amount of use as an energy storage device for storing reserve power) Reinforcement learning can be performed for For example, the agent 154 of the power management device 100 performs reinforcement learning to change the priority charging order of the battery 280 according to the size set for the use of reserve power of the battery 280, and sets a policy accordingly. You can also control to set it up. In the case of the electric vehicle 200 with relatively little or no usage amount (eg, usage amount as an energy storage device for storing reserve power) of the battery 280 of the electric vehicle 200, the power management device 100 The Agent 154 of ) may perform reinforcement learning to lower the charging priority and control to establish a policy accordingly.

In addition, the power management device 100 optimizes the ESS SoC for the load 30 in the area based on statistical information about a certain number or more electric vehicles 200 or electric vehicles 200 operating in a certain area It is possible to perform reinforcement learning related to control of the remaining amount of the battery 280 for adjusting user satisfaction with respect to the electric vehicles 200 and to establish a policy accordingly. For example, the agent 154 of the power management device 100 maintains the SoC of the ESS low in response to the use of the load 30 in an area where the amount of power generation is high but the use of the load 30 is small, while maintaining the electric vehicle 200 Reinforcement learning can be performed to increase the size of the charging amount for each field, and a policy can be established accordingly. Conversely, the agent 154 of the power management device 100 maintains the SoC of the ESS high in response to the use of the load 30 in an area where the amount of power generation is low but the use of the load 30 is high, and the electric vehicle 200 ) can be controlled to perform reinforcement learning to lower the size of the charging amount for the ) and establish a policy accordingly. In addition, the agent 154 of the power management device 100 performs overall control of the electric vehicles 200 and the ESS SoC control according to the load 30 for each region described above, while the specific electric vehicle 200 (eg : An electric vehicle requiring a relatively high charging amount or a special electric vehicle requiring a charging amount higher than the predefined standard charging amount - a special electric vehicle according to user settings or selected by the state or local government, such as a fire engine, police car, ambulance vehicle, etc.) It is possible to selectively increase the amount of charge in the standby state, perform reinforcement learning so that a specific user satisfaction exceeds a predefined value, and establish a policy accordingly.

As set forth above, while this specification contains many specific implementation details, they should not be construed as limiting as to the scope of any invention or claimables, but rather as may be specific to a particular embodiment of a particular invention. It should be understood as a description of the features.

Further, while operations are depicted in the drawings in a specific order, it should not be understood that all illustrated operations must be performed or that those operations must be performed in the specific order shown or in sequential order to obtain desired results. In certain cases, multitasking and parallel processing can be advantageous. Further, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and the program components and systems described may generally be integrated together into a single software product or packaged into multiple software products. You have to understand that you can.

The present description presents the best mode of the invention and provides examples to illustrate the invention and to enable those skilled in the art to make and use the invention. The specification thus prepared does not limit the invention to the specific terms presented. Therefore, although the present invention has been described in detail with reference to the above-described examples, a person skilled in the art may make alterations, changes, and modifications to the present examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be determined by the described embodiments, but by the claims.

10: Power operating environment
100: power management device
110: communication circuit
130: memory
140: display
150: processor
200: electric vehicle
210: vehicle communication circuit
220: sensor unit
230: vehicle memory
240: vehicle display
250: vehicle processor
260: input device
270: means of transportation
280: battery

Claims (10)

A power management method of an energy storage system through operation of an energy storage device for supplying power to a load and a power management device for controlling battery charging of at least one electric vehicle,
collecting home data of an energy storage system related to load in a region where the at least one electric vehicle is located and EV data according to use of the electric vehicle;
storing the EV data and the Home data;
Based on the stored EV data and the Home data, the state of charge of the energy storage system maintains a predefined optimal charge amount corresponding to the load, and the state of charge of the battery of the electric vehicle is strengthened so as to exceed a predefined user satisfaction level. A power management method of an energy storage system comprising the steps of performing learning and establishing a policy accordingly.
According to claim 1,
According to the establishment of the policy, generating an ESS control signal for controlling the state of charge of the energy storage system and a vehicle control signal for controlling the state of charge of the battery of the at least one electric vehicle; Energy characterized in that it further comprises How to operate the power of the storage system.
According to claim 2,
controlling a charge amount of the energy storage system by supplying the ESS control signal to the energy storage system; and
and supplying the vehicle control signal to the at least one electric vehicle to control a battery charge amount of the at least one electric vehicle.
According to claim 3,
The steps to establish the above policy are
Establishing a policy to differently control the charging amount according to the usage time and usage distance of each electric vehicle while maintaining the charged amount of the energy storage system to correspond to the load through the reinforcement learning; How to operate the power of the storage system.
According to claim 4,
Establishing a policy for controlling the charging amount differently according to the usage time and distance
Establishing a policy for controlling a relatively high and fast charging to be performed as the usage time of the electric vehicle approaches and as the usage distance of the electric vehicle increases through the reinforcement learning; or
establishing a policy through the reinforcement learning to control relatively low and slow charging as the usage time of the electric vehicle is longer and the usage distance of the electric vehicle is shorter, through the reinforcement learning; How to operate.
According to claim 3,
The steps to establish the above policy are
In a section in which surplus power of the energy storage system is generated through the reinforcement learning, a battery charging rate and a basic charging amount of the at least one electric vehicle are set relatively high,
Establishing a policy for setting a battery charging rate and a basic charging amount of the at least one electric vehicle relatively low in a period in which peak demand of the energy storage system occurs; How to operate.
a communication circuit for receiving home data about a state of charge of an energy storage system related to a load in an area where at least one electric vehicle is located and EV data according to use of the electric vehicle;
a memory for storing the Home data and the EV data;
A processor functionally connected to the communication circuit and the memory;
The processor
Based on the stored EV data and the Home data, the state of charge of the energy storage system maintains a predefined optimal charge amount corresponding to the load, and the state of charge of the battery of the electric vehicle is strengthened so as to exceed a predefined user satisfaction level. A power management device that supports power management of an energy storage system, characterized in that it is set to perform learning and establish a policy accordingly.
According to claim 7,
The processor
Power supporting power operation of the energy storage system that establishes a policy to have a different charge amount according to the use time and distance of each electric vehicle while maintaining the charge amount of the energy storage system to correspond to the load through the reinforcement learning management device.
According to claim 8,
The processor
Through the reinforcement learning, a policy is established to control relatively high and fast charging as the usage time of the electric vehicle approaches and as the usage distance of the electric vehicle increases, or
The power to support power operation of the energy storage system, characterized in that, through the reinforcement learning, a policy is established to control relatively low and slow charging as the usage time of the electric vehicle is longer and the usage distance of the electric vehicle is shorter. management device.
According to claim 7,
The processor
In a section in which surplus power of the energy storage system is generated through the reinforcement learning, a battery charging rate and a basic charging amount of the at least one electric vehicle are set relatively high,
The power to support power management of the energy storage system, characterized in that to establish a policy to set the battery charging rate and basic charging amount of the at least one electric vehicle relatively low in a section where the peak demand of the energy storage system occurs. management device.