JP7314330B2 - Training method, apparatus, equipment and storage medium for power grid system dispatching model - Google Patents

Description

TECHNICAL FIELD The present application relates to the field of computer technology, in particular to the field of artificial intelligence such as natural language processing and deep learning technology, and specifically to a training method, device, apparatus and storage medium for power grid system dispatching model.

Electric energy is one of the important symbols of modernization and is deeply involved in people's daily life. The grid system is the core force of electricity distribution, playing an important economic and social role by providing reliable electricity to industries and consumers. Affected by unpredictable factors such as sudden events, natural disasters and man-made disasters, the power grid system requires a large number of supervisors and power grid system experts who, along with field knowledge and historical experience, can intervene and maintain different emergency scenes.

From the above, how to increase the degree of automation of grid system dispatching is an urgent problem.

The present application provides a training method, apparatus, apparatus and storage medium for a power grid system dispatching model.

According to one aspect of the present disclosure, obtaining a training data set and a first initial dispatching model, wherein the training data set includes historical execution state information of a power grid system; generating a plurality of first sub-dispatching models based on the first initial dispatching model, each of the first sub-dispatching models being the same as the network structure of the first initial dispatching model; and applying the historical execution state information to each of the first sub-dispatching models. obtaining a first degree of matching between the historical execution state information output by each of the first sub-dispatching models and each candidate operation by inputting; modifying the first initial dispatching model to generate a second initial dispatching model based on the first degree of matching corresponding to each of the plurality of first sub-dispatching models; and executing back to the operation of generating a plurality of first sub-dispatching models based on the second initial dispatching model. and determining that the second initial dispatching model is a grid system dispatching model when a difference between a second matching degree of each candidate action and the historical execution state information determined by the second initial dispatching model and a third matching degree of the historical execution state information and each candidate action determined by the first initial dispatching model falls within a preset range.

According to another aspect of the present application, an apparatus for training a power grid system dispatching model is provided.

A first obtaining module obtains a training data set and a first initial dispatching model, wherein the training data set includes historical running state information of a power grid system, a generating module generates a plurality of first sub-dispatching models based on the first initial dispatching model, each said first sub-dispatching model being the same as the network structure of said first initial dispatching model, and a second obtaining module obtains said historical running state information to each said first sub-dispatching model. to obtain a first degree of matching between the historical execution state information output by each of the first sub-dispatching models and each candidate action, the first training model modifying the first initial dispatching model to generate a second initial dispatching model based on a first degree of matching corresponding to each of the plurality of first sub-dispatching models; and generating a plurality of first sub-dispatching models based on the second initial dispatching model. and determining that the second initial dispatching model is a grid system dispatching model when a difference between a second matching degree of the historical execution state information and each candidate action determined by the second initial dispatching model and a third matching degree of the historical execution state information and each candidate action determined by the first initial dispatching model is within a preset range.

According to another aspect of the present application, a computer apparatus is provided, said computer apparatus including at least one processor and a memory communicatively coupled to said at least one processor, said memory storing instructions executable by said at least one processor, said instructions, when executed by said at least one processor, causing said at least one processor to perform the methods described in the above examples.

According to another aspect of the present application, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing the computer to perform the methods described in the above embodiments.

According to another aspect of the present application, there is provided a computer program which, when executed by a processor, implements the methods described in the above examples.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the application, nor is it intended to limit the scope of the application. Other features of the present application will be readily understood through the following description.

The drawings are used for better understanding of the present technical solution and do not limit the present disclosure.
1 is a flow chart of a method for training a power grid system dispatching model provided by an embodiment of the present application; 4 is a flowchart of another power grid system dispatching model training method provided by an embodiment of the present application; 4 is a flowchart of another power grid system dispatching model training method provided by an embodiment of the present application; 4 is a flowchart of another power grid system dispatching model training method provided by an embodiment of the present application; 1 is a schematic diagram of determining execution actions using a model corresponding to a power grid system provided by an embodiment of the present application; FIG. 4 is a flowchart of another power grid system dispatching model training method provided by an embodiment of the present application; 1 is a schematic diagram of the input and output of a first initial dispatching model provided by an embodiment of the present application; FIG. 4 is a flowchart of another power grid system dispatching model training method provided by an embodiment of the present application; 1 is a schematic diagram of a power grid system dispatching model training process provided by an embodiment of the present application; FIG. 1 is a schematic structural diagram of a power grid system dispatching model training device provided by an embodiment of the present application; FIG. 1 is a block diagram of computer equipment for implementing a method for training a power grid system dispatching model of an embodiment of the present application; FIG.

Illustrative embodiments of the present application are described below in conjunction with the drawings, and various details of the embodiments of the present application are included therein for ease of understanding and should be considered as exemplary only. Accordingly, those skilled in the art should appreciate that various changes and modifications can be made to the examples described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and brevity, the following description omits descriptions of well-known functions and constructions.

Hereinafter, the training method, apparatus, computer equipment and storage medium of the power grid system dispatching model of the embodiments of the present application will be described with reference to the drawings.

Artificial intelligence is a discipline that studies the use of computers to simulate certain human thinking processes and intelligent actions (learning, reasoning, thinking, planning, etc.), and includes hardware-level technology and software-level technology. Artificial intelligence hardware technology generally includes sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology and deep learning, big data processing technology, knowledge graph and other directions.

NLP (Natural Language Processing) is an important direction in the field of computer science and the field of artificial intelligence, and the content of NLP research includes, but is not limited to, branch areas such as text classification, information extraction, automatic summarization, smart question answering, topic recommendation, machine translation, keyword recognition, knowledge base construction, deep text display, named entity recognition, text generation, text analysis (grammar, syntax, grammar, etc.), speech recognition and synthesis, etc. not.

Deep learning is a new research direction in the field of machine learning. Deep learning learns the internal rules and display levels of sample data, and the information obtained in these learning processes greatly aids in the interpretation of data such as text, images, and sounds. The ultimate goal is to enable machines to have the ability to analyze and learn like humans and to recognize data such as text, images, and sounds.

Computer vision is the science that studies how machines "see" and refers to machine vision that uses cameras and computers instead of the human eye to recognize, track, and measure targets, and then perform graphics processing on computers to make images more readily observable by the human eye and suitable for machine detection.

FIG. 1 is a flow chart of a method for training a power grid system dispatching model provided by an embodiment of the present application.

As shown in FIG. 1, the training method for the grid system dispatching model includes the following steps 101-105.

Step 101, obtaining a training data set and a first initial dispatching model, the training data set including historical running state information of the power grid system.

In the present application, historical running state information of the power grid system can be obtained, thereby obtaining a training data set. The historical execution state may be execution state information at a certain time, execution state information within a certain time period, or execution state information within a plurality of time periods.

The running state information of the present application can include, but is not limited to, the active power, reactive power and voltage of the power plant, the active power, reactive power and voltage of the load, the active power, reactive power, voltage and current of the start and end points of the line, the limiting current, the phase structure of the substation, the bus on-off status, and the time information. The time information can include information such as month, week, hour.

When obtaining the training data set, an initial dispatching model may be obtained, which may be referred to as the first initial dispatching model for ease of partitioning. The first initial dispatching model here may be an initial network model, or may have been obtained by pre-training the initial network model.

Step 102, generate a plurality of first sub-dispatching models based on the first initial dispatching model.

In this application, based on the first initial dispatching model, multiple sub-models can be generated, and for ease of partitioning, referred to herein as the first sub-dispatching model, each first sub-dispatching model having the same network structure as the first initial dispatching model.

When generating the plurality of first sub-dispatching models, different Gaussian noise perturbations may be performed on the parameters of the first initial dispatching model to generate the plurality of first sub-dispatching models, for example, by adding noise to the parameters of the first initial dispatching model.

Step 103, input the historical running state information into each first sub-dispatching model to obtain a first matching degree between the historical running state information output by each first sub-dispatching model and each candidate operation.

In the present application, historical execution state information can be input to each first sub-dispatching model, and the first sub-dispatching model is used to process the historical execution state information to obtain a degree of match between the historical execution state information and each candidate action, referred to herein as a first degree of match for ease of partitioning.

There may be multiple candidate actions, and actions can be understood as actions for dispatching the power grid system. For example, operations may include three types of operations: power plant power regulation, bus on/off switching, and substation topological structure change.

The first matching degree of the present application can measure the execution stability when the power grid system executes each candidate action in the historical execution state information, and can be understood as the score of each candidate action predicted by the power grid system in the historical execution state information, the higher the first matching degree, the better the execution stability of the power grid system to execute the corresponding action in the historical execution state information.

For example, the first sub-dispatching model is 200 and the candidate actions are 100. The execution state information at a certain time may be input to each first sub-dispatching model, and each first sub-dispatching model may output a first degree of matching between the execution state information and each candidate action.

When the history execution state information is execution state information within a certain time period, the history execution state information and the first degree of matching between each candidate action include execution state information at each time extracted within the time period and the first degree of matching between each candidate action.

To facilitate the processing of the first sub-dispatching model, the present application may perform normalization pre-processing on the historical execution state information, such as discretization and embedded display on the temporal information.

Step 104, modify the first initial dispatching model to generate a second initial dispatching model based on a first matching degree corresponding to each of the plurality of first sub-dispatching models.

After obtaining the matching degree between the execution state information output by each first sub-dispatching model and each candidate operation, the first initial dispatching model is modified to generate a second initial dispatching model based on the first matching degree corresponding to each of the plurality of first sub-dispatching models.

When modifying, based on the output of each first sub-dispatching model, an operation performed when the power grid system is in the historical running state information can be determined; based on a first degree of matching between the operation and the historical running state information, a parameter adjustment value can be determined; and a second initial dispatching model can be generated by modifying the first initial dispatching model parameter based on the parameter adjustment value.

Step 105, performing back to the operation of generating a plurality of first sub-dispatching models based on the second initial dispatching model, when the difference between the historical execution state information and the second matching degree of each candidate operation determined by the second initial dispatching model and the historical execution state information and the third matching degree of each candidate operation determined by the first initial dispatching model is within a preset range, the second initial dispatching model is the power grid system dispatching model. Decide there is.

After obtaining the second initial dispatching model, a plurality of second sub-dispatching models can be generated based on the second initial dispatching model, and the second sub-dispatching model is the same as the structure of the network of the second initial dispatching model. Then, the historical execution state information is input to each second sub-dispatching model to obtain the degree of matching between the historical execution state information and each candidate operation, the second initial dispatching model is modified based on the corresponding degree of matching for each of the plurality of second sub-dispatching models, and the second initial dispatching model converges to generate a power grid system dispatching model.

The convergence here is that the difference between the historical execution state information and the second matching degree of each candidate action determined by the second initial dispatching model and the historical execution state information and the third matching degree of each candidate action determined by the first initial dispatching model may be within a preset range. That is, the difference between the historical execution state information and the matching degree of each candidate action determined by the current initial dispatching model and the historical execution state information and the matching degree of each candidate action determined by the previous initial dispatching model is within a preset range.

The difference between the second matching degree and the third matching degree here may be the sum of the differences between the second matching degree and the third matching degree corresponding to each candidate action, or the difference between the sum of the second matching degrees of all candidate actions and the sum of the third matching degrees of all actions.

In order to improve the speed of model training, the present application may train in parallel against the first initial dispatching model, for example, the first initial dispatching model contains 5 million parameters, and evolutionary learning can be performed against the first initial dispatching model of 5 million parameters simultaneously on CPU 1000 Plus (Central Processing Unit).

In an embodiment of the present application, based on a first initial dispatching model, generate a plurality of first sub-dispatching models that are the same as the network result, input historical execution state information into each first sub-dispatching model, obtain a first matching degree between the historical execution state information output by each first sub-dispatching model and each candidate operation, and generate a first initial dispatching model based on the first matching degree corresponding to each of the plurality of first sub-dispatching models. Modify, generate a second initial dispatching model, and perform back to the operation of generating a plurality of first sub-dispatching models based on the second initial dispatching model, and if the degree of match output by the second initial dispatching model satisfies the convergence condition, a power grid system dispatching model is obtained. As described above, a power grid system dispatching model can be obtained by performing extensive evolutionary learning on the first initial dispatching model, and the power grid system dispatching model can be used to dispatch the power grid system, thereby improving the degree of automation of the power grid system dispatching.

To improve the accuracy of the model, in one embodiment of the present application, the historical execution state information may include execution state information within multiple time periods, the execution state information within each time period may be interacted with the corresponding first sub-dispatching model, and model training may be performed based on the results of the interaction. Hereinafter, it will be described in conjunction with FIG. 2, which is a flow chart of another power grid system dispatching model training method provided by an embodiment of the present application.

As shown in FIG. 2, the training method for the grid system dispatching model includes the following steps 201-208.

Step 201, obtaining a training data set and a first initial dispatching model, the training data set including historical running state information of the power grid system.

Step 202, generate a plurality of first sub-dispatching models based on the first initial dispatching model.

In the present application, Steps 201 and 202 are the same as Steps 101 and 102 above, so the description is omitted here.

Step 203, inputting the running state information in each time period into the first initial dispatching model to obtain a third matching degree between the running state information in each time period and each candidate operation.

In the present application, the historical execution state information may include execution state information within multiple time periods, such as the execution state information of the power grid system within the first day of a month, the power grid system execution state information within two days, the execution state information of the power grid system within three days, etc.

In the present application, the running state information within each time period can be input into the first initial dispatching model to obtain a third degree of matching between the running state information within each time period and each candidate operation. Here, the third degree of matching between the execution state information in each time period and each candidate action may be the third degree of matching between the execution state information at a certain time in the time period and each candidate action, or may be the third degree of matching between each piece of execution state information at a plurality of times and each candidate action.

Step 204, obtain a first reward value corresponding to the first initial dispatching model for each time period according to the third matching degree corresponding to the first initial dispatching model for each time period.

In this application, the largest third matching degree among the plurality of third matching degrees corresponding to the first initial dispatching model for each time period may be the reward value corresponding to the first initial dispatching model for each time period, and may be referred to as the first reward value for ease of segmentation. Alternatively, the sum of the third degree of matching between the execution state information in each time period and each candidate action output by the first initial dispatching model can be set as the first reward value corresponding to the first initial dispatching model in each time period.

Step 205, inputting the running state information within each time period into the corresponding first sub-dispatching model to obtain a first matching degree between the running state information within each time period and each candidate operation.

In the present application, the execution state information in each time period can be input to the corresponding first sub-dispatching model to obtain a first degree of matching between the execution state information of each time period output by the corresponding first sub-dispatching model and each candidate operation.

In other words, the time period for inputting the execution state information of each first sub-dispatching model is different.

In the present application, the correspondence between the time period and each first sub-dispatching model may be set as required or determined randomly. For example, the order before and after the time period may be determined, and the execution state information for each time period may be input to the first sub-dispatching model in ascending order of number.

Execution state information for one time period is randomly selected and input to the first sub-dispatching model.

Step 206, obtain a second reward value corresponding to the first sub-dispatching model corresponding to each time period according to the first matching degree corresponding to the first sub-dispatching model corresponding to each time period.

In the present application, step 206 is similar to step 204 above, and therefore will not be described here.

Step 207, modify the first initial dispatching model to generate a second initial dispatching model according to the first reward value and the second reward value corresponding to each of the plurality of time periods.

For each time slot, the first reward value corresponding to the first sub-dispatching model may be subtracted from the second reward value corresponding to the second sub-dispatching model to obtain a reward value after normalization of the second sub-dispatching model within each time slot. That is, the difference between the reward value corresponding to the first sub-dispatching model and the reward value corresponding to the first initial dispatching model within the same time slot can be the reward value after the first sub-dispatching model is normalized.

After obtaining the normalized reward value corresponding to each first sub-dispatching model, integration such as addition may be performed on the normalized reward value corresponding to each of the plurality of first sub-dispatching models, determining network parameter adjustment values based on the integrated reward values, and using the adjustment values to adjust the parameters of the first initial dispatching model to generate a second initial dispatching model.

In the present application, the evolution direction of the network parameters of the first initial network model can be determined based on the normalized reward values corresponding to each of the plurality of first sub-dispatching models, thereby modifying the first initial dispatching model and generating a second initial dispatching model.

Step 208, performing back to the operation of generating a plurality of first sub-dispatching models based on the second initial dispatching model, when the difference between the historical execution state information and the second matching degree of each candidate operation determined by the second initial dispatching model and the historical execution state information and the third matching degree of each candidate operation determined by the first initial dispatching model is within a preset range, the second initial dispatching model is the power grid system dispatching model. Decide there is.

In the present application, step 208 is similar to step 105 above, and therefore will not be described here.

In an embodiment of the present application, the historical state information may include execution state information within a plurality of time periods, the execution state information within each time period may be input into a first initial dispatching model to obtain a third degree of matching between the execution state information within each time period and each candidate action, determining a first reward value corresponding to the first initial dispatching of each time period based on the third degree of matching corresponding to the first initial dispatching of each time period; Inputting the execution state information into a corresponding first sub-dispatching model to obtain a first degree of matching between the execution state information and each candidate action within each time period, determining a second reward value corresponding to the first sub-dispatching model corresponding to each time period based on the first matching degree corresponding to the first sub-dispatching model corresponding to each time period, and obtaining a first initial dispatching model based on the first reward value and the second reward value corresponding to each of a plurality of times. , generate a second initial dispatching model to continue training, and finally generate a grid system dispatching model. Thus, each of the first sub-dispatching models interacts with the power grid system in different time periods to train the first initial dispatching model and improve the accuracy of the model.

In one embodiment of the present application, the first reward value can be obtained according to the scheme shown in FIG. FIG. 3 is a flowchart of another grid system dispatching model training method provided by an embodiment of the present application.

As shown in FIG. 3, the step of obtaining a first reward value corresponding to the first initial dispatching model for each time period includes steps 301-304 as follows.

Step 301, extracting execution state information at a plurality of times from the execution state information within each time period.

In the present application, it is possible to extract execution state information for a plurality of times from execution state information for each time period. For example, it is possible to extract 1000 times of execution state information from the execution state information of the power grid system on a certain day.

Step 302, inputting the execution state information of each time into the first initial dispatching model to obtain a third matching degree between the execution state information of each time and each candidate operation.

After obtaining the execution state information of a plurality of times, the execution state information of each time can be input to the first initial dispatching model, and a third degree of matching between the execution state information of each time and each candidate operation can be obtained. That is, it is possible to input the execution state information at each time into the first initial dispatching model and obtain the score of each candidate action in the execution state information at each time.

Step 303, extract a first target action from each candidate action according to each third matching degree.

For execution state information at each time, a first target action can be extracted from each candidate action based on a third degree of matching between the execution state information at each time and each candidate action. Accordingly, the corresponding first target action can be acquired based on the execution state information at each time.

In the present application, a candidate motion with the third highest degree of matching can be extracted as the first target motion from the plurality of candidate motions.

Step 304, determine a first reward value based on a third degree of matching between each of the execution state information at the plurality of times and the first target action.

After extracting the first target action based on the third degree of matching between the execution state information at each time and each candidate action, a first reward value can be determined based on the first degree of matching between each of the execution state information at a plurality of times and the first target action.

For example, a sum of first degrees of matching corresponding to all first target actions may be the first reward value. That is, for the execution state information at each time within a certain time slot, the action performed by the power grid system can be determined based on the output of the first initial dispatching model, and the sum of the third matching degrees corresponding to the actions determined each time within the time slot is taken as the first reward value.

Alternatively, the model corresponding to the power grid system may be controlled to be executed based on the acquired first target action for the execution state information at each time point within a certain time period. Based on the execution state, the score of the first target action is determined, and the sum of the points of the first target actions corresponding to all times within the time period is set as the first reward value.

It should be noted that when obtaining the second reward value, the same method as in FIG. 3 may be used to obtain it, so the description is omitted here.

In an embodiment of the present application, when obtaining a first reward value corresponding to a first initial dispatching model in each time period, the execution state information at a plurality of times is extracted from the execution state information in each time period, the execution state information at each time is input to the first initial dispatching model, a third degree of matching between the execution state information at each time and each candidate action is obtained, a first target action is extracted from the candidate action, and a first target action is extracted from each of the execution state information at a plurality of times and the first target action. A first reward value is determined based on the third degree of match. As described above, it is possible to determine the first reward value based on the degree of matching corresponding to the first target action determined by accumulating a plurality of times within the time period.

In the above example, the first target action may be extracted directly based on the third matching degree, and in one embodiment of the present application, the first target action may be extracted based on the execution state of the model corresponding to the power grid system, the determined matching degree. 4, which is a flowchart of another power grid system dispatching model training method provided by an embodiment of the present application.

As shown in FIG. 4, the step of extracting a first target motion from a plurality of candidate motions based on the respective third matching degrees includes steps 401-403 below.

Step 401, extracting a plurality of reference motions from each candidate motion according to each third matching degree.

In the present application, a plurality of motions can be extracted from a plurality of candidate motions based on a third degree of matching in which the execution state information of each time corresponds to each candidate motion, respectively, for the execution state information of each time, and are referred to herein as reference motions.

Step 402, according to each reference action, control the model corresponding to the power grid system to execute, and determine a first reference matching degree between the execution state information at each time and each reference action according to the execution state of the model.

In the present application, the execution state information at each time may be input to the model corresponding to the power grid system, and based on each reference operation, the model is controlled to be executed, and the degree of matching between the execution state information at each time and each reference operation is determined based on the execution state of the model. For ease of segmentation, the model corresponding to the grid system, called the first degree of reference match, may be a pre-built simulation model of the grid system based on expert knowledge.

For ease of understanding, the execution state information at a certain time may be regarded as one scene, and for each execution scene, based on each reference action, it is possible to control the model corresponding to the power grid system to execute, and thus, based on the execution state of the model, the first reference degree of each scene and each reference action can be determined.

In practical applications, the actions to be performed may be selected based on a model corresponding to the power grid system. As shown in FIG. 5, taking the case of whether the bus of the power grid system is overloaded or not, it is determined whether the bus of the power grid system is overloaded. If there is a situation in which the power grid system is overloaded, the power grid system can be controlled to execute a model corresponding to the power grid system according to each candidate operation, and based on the model execution result, the operation with the highest score (i.e., matching degree) can be selected and executed, and then enter the next state. If the grid system does not have a bus overload situation, it will enter the next state directly without action.

Step 403, extract a first target action from the plurality of reference actions according to each first reference matching degree.

After determining the first reference matching degree between the execution state information at each time and each reference action, the action having the highest first reference matching degree among the plurality of reference actions is set as the first target action.

In an embodiment of the present application, when extracting the first target action, a plurality of reference actions may be extracted from each candidate action based on a third degree of matching determined by the first initial dispatching model, and a first target action may be extracted from the plurality of reference actions based on a model corresponding to the power grid system. As described above, the first target operation corresponding to the execution state information at each time is determined based on the first initial dispatching model and the model corresponding to the power grid system, thereby improving the accuracy of determining the first target operation.

In one embodiment of the present application, the method shown in FIG. 6 can be trained to obtain a first initial dispatching model. FIG. 6 is a flowchart of another grid system dispatching model training method provided by an embodiment of the present application.

As shown in FIG. 6, before obtaining the training data set and the first initial dispatching model, the method further includes the following steps 601-603.

Step 601, based on each candidate action, control the model corresponding to the power grid system to execute, and determine a second reference matching degree between the execution state information at each time and each candidate action.

In the present application, the execution state at multiple times can be obtained in advance as a training data set. After obtaining the execution state information at a plurality of times, the model corresponding to the power grid system is controlled to execute based on each candidate operation, and a second reference match degree between the execution state information at each time and each candidate operation can be determined based on the execution state of the model.

Step 602, inputting the execution state information of each time into the initial network model to obtain a fourth degree of matching between the execution state information of each time and each candidate operation.

In the present application, the execution state information at each time can be input into the initial network model, and the initial network model can be used to process the execution state information at each time to obtain a fourth degree of matching between the execution state information at each time and each candidate operation. That is, each candidate action can obtain a score in the execution state information at each time.

Assuming that the number of candidate actions is N, as shown in FIG. 7, execution state information at a certain time is input to the model, and the model can input scores from action 1 to action N, and the scores here can predict the degree of matching between the execution state information and actions at that time.

Step 603, modify the initial network model according to the difference between each fourth matching degree and the corresponding second reference matching degree in the execution state information at each time, and determine that the modified initial network model is the first initial dispatching model when the difference between the execution state information at each time determined by the modified initial network model and the fourth matching degree of each candidate operation and the second reference matching degree is within a preset range.

In the present application, the initial network model is modified based on the difference between each fourth matching degree and the corresponding second reference matching degree in the running state information at each time, and when the difference between the running state information at each time determined by the modified initial network model and the fourth matching degree of each candidate operation and the second reference matching degree is within a preset range, continue training using the modified initial network model, and determine that the modified initial network model is the first initial dispatching model. .

Here, the fact that the difference between the fourth matching degree and the second reference matching degree between the execution state information at each time and each candidate action is within a preset range may be that the difference between the fourth matching degree corresponding to each candidate action and the second reference matching degree is both within a preset range, and the difference between the sum of the fourth matching degrees corresponding to all candidate actions and the sum of the second reference matching degrees corresponding to all candidate actions is within a preset range. It can be something.

In the present application, when training the first initial dispatching model, a deep learning method may be adopted for training.

In an embodiment of the present application, before obtaining the training data set and the first initial dispatching model, the model corresponding to the power grid system can be controlled to execute according to each candidate operation, determine a second reference matching degree between the execution state information at each time and each candidate operation, input the execution state information at each time into the initial network model, obtain a fourth matching degree between the execution state information at each time and each candidate operation, and obtain a fourth matching degree between the execution state information at each time and each candidate operation; An initial network model is trained to generate a first initial dispatching model based on the difference between the 4 degree of match and the reference degree of match. Thus, by using the reference matching degree obtained by the simulation model built on the basis of the expert's knowledge, the expert's knowledge is combined with the first initial dispatching model obtained by training, and the training is continued to obtain the first initial dispatching model, followed by training to obtain the power grid system dispatching model, not only improving the training speed of the power grid system dispatching model, but also improving the accuracy of the model.

In practical applications, due to the relatively complex topological structure of a typical power grid system, the number of dispatchable operations of the power grid system is very large. In one embodiment of the present application, during the process of training the initial network model to obtain the first initial dispatching model, prior to determining the second degree of reference match between the execution state information at each time and each candidate operation, operations with high execution frequency can be selected as candidate operations from a large number of operations. Hereinafter, it will be described in conjunction with FIG. 8, which is a flow chart of another grid system dispatching model training method provided by an embodiment of the present application.

As shown in FIG. 8, the following steps 801-803 are further included before determining the second degree of reference match between the execution state information at each time and each candidate operation.

Step 801: Based on each action, control the model corresponding to the power grid system to execute, and determine a third reference matching degree between the execution state information at each time and each action.

In the present application, step 801 is the same as step 601 above, so the description is omitted here.

Step 802, based on each third degree of reference matching, determine the operation with the highest degree of third reference matching with the execution state information at each time.

In the present application, based on the third degree of reference match between the execution state information at each time and each action, the action having the highest third degree of reference match with the execution state information at each time can be determined.

Step 803: Determine the number of times of the highest third reference match for each action based on the action with the highest third reference match with the execution state information at each time.

After determining the operation with the highest third reference matching degree with the execution state information at each time, based on the operation with the highest third reference matching degree with the execution state information at each time, the number of times of the highest third reference matching degree of each operation can be determined.

When the execution state information of one time is regarded as a scene, the highest third reference match count for each action can be determined based on the highest third reference match determined in each scene.

Step 804, extract a plurality of candidate motions from each motion based on the highest number of third reference matching degrees of each motion.

In the present application, a motion with the highest third reference matching degree count greater than a threshold may be a candidate motion.

In an embodiment of the present application, before determining a second degree of reference match between the execution state information at each time and each candidate operation, the model corresponding to the power grid system can be controlled to execute based on each operation, determine a third degree of reference match between the execution state information at each time and each operation, and select a plurality of candidate operations from each operation based on the third degree of reference match corresponding to each operation in the execution state information at each time. As described above, by using a simulation model constructed based on the knowledge of an expert, motions that are frequently executed may be selected from a large number of motions and used as candidate motions.

FIG. 9 is a schematic diagram of a power grid system dispatching model training process provided by an embodiment of the present application.

As shown in FIG. 9, one neural network model can be subjected to noise perturbation, resulting in n+1 sub-models with noise,

For each sub-model, the running state information within the corresponding time period can be input into the sub-model to obtain the normalized reward value corresponding to the sub-model. for example,

denotes the second reward value corresponding to the initial dispatching model. The normalized reward values corresponding to the rest of the sub-models are similar and are omitted here.

After obtaining the normalized reward values corresponding to each of the n+1 submodels, a new initial dispatching model can be generated based on the n+1 normalized reward values.

In one embodiment of the present application, after obtaining the grid system dispatching model, the grid system dispatching model may be used to perform grid system dispatching.

In the present application, the current running state information of the power grid system can be obtained, and the current running state information is input to the power grid system dispatching model to obtain the degree of matching between the current running state information output by the power grid system dispatching model and each candidate operation.

After obtaining the degree of matching between the current execution state information and each candidate action, a second target action can be extracted from each candidate action based on the degree of matching between the current execution state information and each candidate action. For example, the candidate operation with the highest degree of matching may be directly selected as the second target operation, or a plurality of operations may be selected from each candidate operation, and based on each selected operation, the model corresponding to the power grid system may be controlled to be executed, the degree of matching between each selected operation and the current execution state information may be determined, and the operation with the highest matching degree may be selected as the second target operation. After determining the second target behavior, the power grid system can be dispatched based on the second target behavior.

For example, the number of candidate operations is 100, and based on the matching degree output by the power grid system dispatching model, the top 20 operations with the highest matching degree can be extracted, and based on the matching degree obtained by the model corresponding to the power grid system, the operation with the highest matching degree with the current execution state information is extracted from among them, and the power grid system dispatching is performed.

In an embodiment of the present application, after determining that the second initial dispatching model is a grid system dispatching model, the current running state information of the power grid system may be input into the grid system dispatching model to obtain a degree of match between the current running state information and each candidate action, and based on the obtained degree of match corresponding to each candidate action, determine an action for grid system dispatching. From the above, the grid system dispatching model is used to determine the current running state information, perform the grid system dispatching operation, and improve the automation degree of the grid system dispatching.

To implement the above embodiments, the embodiments of the present application further provide a training device for power grid system dispatching model. FIG. 10 is a schematic structural diagram of a training device for power grid system dispatching model provided by an embodiment of the present application.

As shown in FIG. 10 , the power grid system dispatching model training device 1000 includes a first acquisition module 1010 for acquiring a training data set and a first initial dispatching model, wherein the training data set includes historical running state information of the power grid system, and a generation module 1020 for generating a plurality of first sub-dispatching models based on the first initial dispatching model, each first sub-dispatching model. a generation module 1020 whose processing model is the same as the network structure of the first initial dispatching model; a second acquisition module 1030 that inputs the historical execution state information to each of the first sub-dispatching models to acquire a first degree of matching between the history execution state information output by each of the first sub-dispatching models and each candidate action; modifying the dispatching model to generate a second initial dispatching model; returning to the operation of generating a plurality of first sub-dispatching models based on the second initial dispatching model; , a first training model 1040 that determines that the second initial dispatching model is a grid system dispatching model.

In one possible implementation of an embodiment of the present application, the historical state information includes execution state information within multiple time periods, and the second obtaining module 1030 inputs the execution state information within each time period into a corresponding first sub-dispatching model to obtain a first degree of matching between the execution state information within each time period and each of the candidate operations.

The first training module 1040 includes: a first acquisition unit that inputs execution state information in each time period to the first initial dispatching model to obtain a third matching degree between the execution state information in each time period and each candidate action; a second acquisition unit that obtains a first reward value corresponding to the first initial dispatching model in each time period based on the third matching degree corresponding to the first initial dispatching model in each time period; the second obtaining unit obtaining a second reward value corresponding to the corresponding first sub-dispatching model for each time period based on a first degree of matching corresponding to the corresponding first sub-dispatching model; and a training unit modifying the first initial dispatching model to generate the second initial dispatching model based on the first reward value and the second reward value corresponding to each of the plurality of time periods.

In one possible implementation of an embodiment of the present application, the first obtaining unit extracts execution state information at multiple times from the execution state information within each time period, inputs the execution state information at each time into the first initial dispatching model, and obtains a third degree of matching between the execution state information at each time and each candidate operation.

The second obtaining unit further extracts a first target action from each of the candidate actions based on each of the third degrees of matching, and determines the first reward value based on a third degree of matching between each of the execution state information at the plurality of times and the first target action.

In one possible implementation of an embodiment of the present application, the second obtaining unit further extracts a plurality of reference actions from each of the candidate actions based on each of the third matching degrees, controls a model corresponding to the power grid system to execute based on each of the reference actions, determines a first reference match degree between the execution state information at each time and each of the reference actions based on the execution state of the model, and determines, based on each of the first reference match degrees, among the plurality of reference actions. extracting the first target motion;

In one possible implementation of an embodiment of the present application, the apparatus controls a model corresponding to the power grid system to execute, based on each candidate action, to determine a second degree of reference match between the execution state information of each time and each of the candidate actions, a third obtaining module to input the execution state information of each time into an initial network model to obtain a fourth degree of match between the execution state information of each time and each of the candidate actions, in the execution state information of each time: a second training module for modifying the initial network model based on a difference between each of the fourth matching degrees and the corresponding second reference matching degrees, and determining that the modified initial network model is the first initial dispatching model when the difference between the execution state information at each time determined by the modified initial network model and the fourth matching degree of each candidate action and the second reference matching degrees is within a preset range.

In one possible implementation of an embodiment of the present application, the first determination module further controls a model corresponding to the power grid system to execute, based on each action, to determine a third degree of reference match between each time execution state information and each said action.

The apparatus comprises: a second determination module that determines an action having the highest third reference match degree with the execution state information at each time based on each of the third reference match degrees; a third determination module that determines the number of times each of the actions with the highest third reference match degree based on the action with the highest third reference match degree with the execution state information at each time; 1 extraction module.

In one possible implementation of an embodiment of the present application, the apparatus comprises: a fourth obtaining module for obtaining current running state information of the power grid system; a fifth obtaining module for inputting the current running state information to the power grid system dispatching model to obtain a degree of match between the current running state information and each of the candidate actions; a second extraction module for extracting a second target action from each of the candidate actions based on a degree of match between the current running state information and each of the candidate actions; a dispatching module for dispatching the power grid system based on:

In addition, the description of the embodiment of the training method of the power grid system dispatching model is also applicable to the training apparatus of the power grid system dispatching model of the embodiment, so the description is omitted here.

In an embodiment of the present application, based on a first initial dispatching model, generate a plurality of first sub-dispatching models that are the same as the network result, input historical execution state information into each first sub-dispatching model, obtain a first matching degree between the historical execution state information output by each first sub-dispatching model and each candidate operation, and generate a first initial dispatching model based on the first matching degree corresponding to each of the plurality of first sub-dispatching models. Modify, generate a second initial dispatching model, and perform back to the operation of generating a plurality of first sub-dispatching models based on the second initial dispatching model, and if the degree of match output by the second initial dispatching model satisfies the convergence condition, a power grid system dispatching model is obtained. From the above, a grid system dispatching model can be obtained by performing extensive evolutionary learning on the first initial dispatching model, and dispatching the grid system using the grid system dispatching model can improve the degree of automation of the grid system dispatching.

According to embodiments of the present application, the present application further provides a computer apparatus, a readable storage medium, and a computer program product.

FIG. 11 shows a schematic block diagram of an exemplary electronic computing device 1100 for implementing embodiments of the present application. Computer equipment is intended to represent various forms of digital computers such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Computing devices can also represent various forms of mobile devices such as personal digital assistants, mobile phones, smart phones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functionality illustrated herein are merely examples and are not intended to limit the description and/or required implementation of the application herein.

As shown in FIG. 11, the device 1100 includes a computing unit 1101 that performs various suitable operations and processes in accordance with computer programs stored in read-only memory (ROM) 1102 or computer programs loaded from storage unit 1108 into random access memory (RAM) 1103. Various programs and data necessary for the operation of the device 1100 may also be stored in the RAM 1103 . Computing unit 1101 , ROM 1102 and RAM 1103 are connected to each other via bus 1104 . An I/O (Input/Output) interface 1105 is also connected to the path.

Several components of the device 1100 are connected to the I/O interface 1105, including an input unit 1106 such as a keyboard, mouse, etc., an output unit 1107 such as each type of display, speakers, etc., a storage unit 1108 such as a magnetic disk, an optical disk, etc., and a communication unit 1109 such as a network card, modem, wireless communication transceiver. Communications unit 1109 enables device 1100 to exchange information/data with other devices via computer networks such as the Internet and/or various telegraph networks.

Computing unit 1101 may be various general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of computational unit 1101 include a CPU (Central Processing Unit), a GPU (Graphic Processing Unit) (GPU), various dedicated AI (Artificial Intelligence) computational chips, various machine-executed learning model algorithm computational units, a DSP (Digital Sign al Processor, digital signal processor), and any suitable processor, controller, microcontroller, or the like. The computing unit 1101 performs each of the methods and processes described above, eg, the method of training the grid system dispatching model. For example, in some embodiments a method for training a power grid system dispatching model may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108 . In some embodiments, some or all of the computer programs may be loaded and/or installed on device 1100 via ROM 1102 and/or communication unit 1109 . When the computer program is loaded into RAM 1103 and executed by computing unit 1101, one or more steps of the method for training a grid system dispatching model described above may be performed. Alternatively, in other embodiments, computing unit 1101 may be configured in any other suitable manner (eg, via firmware) to perform the grid system dispatching model training method.

Various embodiments of the systems and techniques described herein above include digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Integrated Circuits). Application Specific Standard Product), SOC (System On Chip), CPLD (Complex Programmable Logic Device), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being embodied in one or more computer programs, which may be executed and/or interpreted in a programmable system including at least one programmable processor, which may be an application-specific or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, at least one input device, and at least one output device. I can.

Program code to implement the methods of the present application may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that when executed by the processor or controller, perform the functions/acts specified by the flowcharts and/or block diagrams. The program code may execute entirely on a machine, partially on a machine, as a standalone software package, partially on a machine, and partially on a remote machine, or entirely on a remote machine or server.

In the context of this application, a machine-readable medium may be a tangible medium capable of containing or storing a program for use by or in conjunction with an instruction execution system, apparatus, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of any of the foregoing. More specific examples of machine-readable storage media are electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory). memory) including optical storage, magnetic storage, or any suitable combination of any of the foregoing.

The systems and techniques described herein can be implemented on a computer to provide interaction with a user, the computer having a display device (e.g., a cathode-ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball), the user interacting with the keyboard and Input can be provided to the computer by the pointing device. Other types of devices can also provide interaction with a user, for example, the feedback provided to the user can be any form of sensing feedback (e.g., visual, auditory, or tactile feedback) and can receive input from the user in any form (including acoustic, speech, or tactile input).

The systems and techniques described herein can be executed on a computing system that includes back-end components (e.g., a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or any combination of such back-end components, middleware components, and front-end components. I can. The components of the system can be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, blockchain networks.

The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship to each other. The server may be a cloud server, also called a cloud computing server or cloud host, which is one host product in the cloud computing service system, and solves the defects of the conventional physical host and VPS service (Virtual Private Server), which are difficult to manage and weak in business extensibility. The server may be a server of a distributed system, or a server combined with a blockchain.

According to an embodiment of the present application, the present application further provides a computer program, the instructions of the computer program, when executed by a processor, perform the power grid system dispatching model training method provided by the above-described embodiment of the present application.

It should be appreciated that steps may be reordered, added, or deleted using the various forms of flow shown above. For example, each step described in the present disclosure may be performed in parallel, sequentially, or in a different order, but is not limited herein as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above specific embodiments do not limit the protection scope of this application. Those skilled in the art can make various modifications, combinations, subcombinations, and substitutions depending on design requirements and other factors. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall all fall within the protection scope of this application.

Claims (17)

A method of training a power grid system dispatching model, comprising:
obtaining a training data set and a first initial dispatching model, the training data set including historical running state information of a power grid system;
generating a plurality of first sub-dispatching models based on said first initial dispatching model, each said first sub-dispatching model being the same as the network structure of said first initial dispatching model;
inputting the historical execution state information to each of the first sub-dispatching models to obtain a first degree of matching between the historical execution state information output by each of the first sub-dispatching models and each candidate action;
modifying the first initial dispatching model to generate a second initial dispatching model based on a first degree of match corresponding to each of the plurality of first sub-dispatching models;
returning to the step of generating a plurality of first sub-dispatching models based on the second initial dispatching model, wherein when a difference between a second degree of matching of each candidate action and the historical execution state information determined by the second initial dispatching model and a third degree of matching of the historical execution state information and each candidate action determined by the first initial dispatching model falls within a preset range, the second initial dispatching model is adapted to the power grid system dispatching model; determining to be a model;
A training method for a power grid system dispatching model, characterized by: wherein the historical execution state information includes execution state information within a plurality of time periods, and obtaining a first degree of matching between the historical execution state information output by each of the first sub-dispatching models and each candidate action;
inputting the execution state information within each time slot into a corresponding first sub-dispatching model to obtain a first degree of matching between the execution state information within each time slot and each of the candidate actions;
modifying the first initial dispatching model to generate a second initial dispatching model based on a first degree of match corresponding to each of the plurality of first sub-dispatching models;
inputting execution state information within each time period into the first initial dispatching model to obtain a third degree of matching between the execution state information within each time period and each candidate action;
obtaining a first reward value corresponding to the first initial dispatching model for each time slot based on a third degree of match corresponding to the first initial dispatching model for each time slot;
obtaining a second reward value corresponding to the corresponding first sub-dispatching model for each time slot based on a first degree of match corresponding to the corresponding first sub-dispatching model for each time slot;
modifying the first initial dispatching model to generate the second initial dispatching model based on a first reward value and a second reward value corresponding to each of the plurality of time slots;
2. The method of claim 1, wherein: The step of inputting execution state information within each time period into the first initial dispatching model to obtain a third degree of matching between the execution state information within each time period and each candidate action,
a step of extracting execution state information at a plurality of times from the execution state information within each time period;
inputting the execution state information at each time into the first initial dispatching model to obtain a third degree of matching between the execution state information at each time and each candidate action;
obtaining a first reward value corresponding to the first initial dispatching model for each time period based on a third degree of match corresponding to the first initial dispatching model for each time period,
extracting a first target action from each of said candidate actions based on each said third degree of matching;
determining the first reward value based on a third degree of matching between each of the plurality of times of execution state information and a first target action;
3. The method of claim 2, wherein: extracting a first target action from a plurality of candidate actions based on each said third degree of matching;
extracting a plurality of reference motions from each of the candidate motions based on each of the third degrees of matching;
a step of controlling a model corresponding to the power grid system to be executed based on each of the reference actions, and determining a first reference matching degree between the execution state information at each time and each of the reference actions based on the execution state of the model;
extracting the first target action from the plurality of reference actions based on each of the first reference match degrees;
4. The method of claim 3, wherein: Before the step of obtaining the training dataset and the first initial dispatching model,
Based on each candidate action, controlling a model corresponding to the power grid system to execute, and determining a second reference match degree between the execution state information at each time and each of the candidate actions;
inputting the execution state information at each time into an initial network model to obtain a fourth degree of matching between the execution state information at each time and each of the candidate actions;
modifying the initial network model based on the difference between each fourth matching degree and the corresponding second reference matching degree in the execution state information at each time, and determining that the modified initial network model is the first initial dispatching model when the difference between the execution state information at each time and the fourth matching degree of each candidate action determined by the modified initial network model and the second reference matching degree is within a preset range;
2. The method of claim 1, wherein: before the step of determining a second degree of reference match between execution state information at each time and each said candidate action;
controlling a model corresponding to the power grid system to execute based on each action to determine a third reference match degree between the execution state information at each time and each of the actions;
determining an operation having the highest third degree of reference match with the execution state information at each time based on each degree of third reference match;
determining the number of times each of the actions with the highest third degree of reference matching is performed based on the action with the highest third degree of reference matching with the execution state information at each time;
extracting a plurality of candidate actions from each action based on the number of times of each said action with the highest third degree of reference match;
6. The method of claim 5, wherein: After determining that the second initial dispatching model is a grid system dispatching model,
obtaining current running state information of the power grid system;
inputting the current running state information into the power grid system dispatching model to obtain a degree of match between the current running state information and each of the candidate actions;
extracting a second target action from each of the candidate actions based on the degree of matching between the current execution state information and each of the candidate actions;
dispatching the power grid system based on the second target action;
The method according to any one of claims 1 to 6, characterized in that: A power grid system dispatching model training device comprising:
a first acquisition module for acquiring a training data set and a first initial dispatching model, wherein the training data set includes historical running state information of a power grid system;
a generation module for generating a plurality of first sub-dispatching models based on the first initial dispatching model, each of the first sub-dispatching models having the same network structure as the first initial dispatching model;
a second obtaining module for inputting the historical execution state information to each of the first sub-dispatching models to obtain a first degree of matching between the historical execution state information output by each of the first sub-dispatching models and each candidate action;
前記複数の第１のサブディスパッチングモデルのそれぞれに対応する第１のマッチ度に基づいて、前記第１の初期ディスパッチングモデルを修正して、第２の初期ディスパッチングモデルを生成し、前記第２の初期ディスパッチングモデルに基づいて、複数の第１のサブディスパッチングモデルを生成する前記生成モジュールの操作に戻って実行し、前記第２の初期ディスパッチングモデルによって決定された前記履歴実行状態情報及び各候補動作の第２のマッチ度と、前記第１の初期ディスパッチングモデルによって決定された前記履歴実行状態情報及び各候補動の第３のマッチ度との差が、予め設定された範囲内になると、前記第２の初期ディスパッチングモデルが送電網システムディスパッチングモデルであると決定する第１のトレーニングモジュールと、を含む、
A power grid system dispatching model training device characterized by: The historical execution state information includes execution state information within a plurality of time periods, and the second acquisition module comprises:
inputting the execution state information within each time slot into a corresponding first sub-dispatching model to obtain a first matching degree between the execution state information within each time slot and each of the candidate actions;
The first training module comprises:
a first obtaining unit for inputting execution state information within each time period into the first initial dispatching model to obtain a third degree of matching between the execution state information within each time period and each candidate action;
a second obtaining unit for obtaining a first reward value corresponding to the first initial dispatching model for each time period based on a third matching degree corresponding to the first initial dispatching model for each time period, and further obtaining a second reward value corresponding to the corresponding first sub dispatching model for each time period based on the first matching degree corresponding to the corresponding first sub dispatching model for each time period;
a training unit that modifies the first initial dispatching model to generate the second initial dispatching model based on a first reward value and a second reward value corresponding to each of the plurality of time slots;
9. Apparatus according to claim 8, characterized in that: The first acquisition unit comprises:
Extract execution state information for multiple times from the execution state information within each time period,
inputting the execution state information at each time into the first initial dispatching model to obtain a third degree of matching between the execution state information at each time and each candidate action;
The second acquisition unit further comprises:
extracting a first target action from each of the candidate actions based on each of the third degrees of matching;
determining the first reward value based on a third degree of matching between each of the execution state information at the plurality of times and the first target action;
10. Apparatus according to claim 9, characterized in that: The second acquisition unit further comprises:
extracting a plurality of reference motions from each of the candidate motions based on each of the third matching degrees;
controlling a model corresponding to the power grid system to be executed based on each of the reference actions, determining a first reference matching degree between the execution state information at each time and each of the reference actions based on the execution state of the model;
extracting the first target action from the plurality of reference actions based on each of the first reference matching degrees;
11. Apparatus according to claim 10, characterized in that: a first determination module for controlling a model corresponding to the power grid system to execute based on each candidate action to determine a second reference match degree between execution state information at each time and each of the candidate actions;
a third acquisition module that inputs the execution state information at each time into an initial network model to acquire a fourth degree of matching between the execution state information at each time and each of the candidate actions;
In the execution status information of each time, based on the difference between the fourth match between each time and the degree of reference matching in correspondence, each time the above -time network model is corrected and the modified initial network model is determined by the modified initial network model, and the degree of matching degree of the second reference match between each time each time and each candidate operation. When it is within the set in advance, it includes the second training module that determines that the modified initial network model is the first initial dispatching model.
9. Apparatus according to claim 8, characterized in that: The first determination module further controls a model corresponding to the power grid system to execute, based on each action, to determine a third reference match degree between the execution state information at each time and each of the actions;
The device comprises:
a second determination module that determines an operation having the highest third degree of reference match with the execution state information at each time based on each degree of third reference match;
a third determination module for determining the number of times each of the actions having the highest third reference matching degree with the execution state information at each time based on the action having the highest third reference matching degree;
a first extraction module for extracting a plurality of candidate actions from each action based on the number of times each said action with the highest third degree of reference match;
13. Apparatus according to claim 12, characterized in that: a fourth acquisition module for acquiring current running state information of the power grid system;
a fifth acquisition module for inputting the current running state information into the power grid system dispatching model to acquire a degree of match between the current running state information and each of the candidate actions;
a second extraction module for extracting a second target action from each of the candidate actions based on the degree of matching between the current execution state information and each of the candidate actions;
a dispatching module that dispatches the power grid system based on the second target action;
A device according to any one of claims 8 to 13, characterized in that: a computer device,
at least one processor;
a memory communicatively coupled to the at least one processor;
Instructions executable by the at least one processor are stored in the memory, and the instructions, when executed by the at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 7,
computer equipment. A non-transitory computer-readable storage medium having computer instructions stored thereon,
The computer instructions cause the computer to perform the method of any one of claims 1-7,
A non-transitory computer-readable storage medium. A computer program,
The computer program, when executed by a processor, implements the method of any of claims 1-7 ,
computer program.

Images