CN114138196A - Power system data storage method and device, computer equipment and storage medium - Google Patents
Power system data storage method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114138196A CN114138196A CN202111422631.9A CN202111422631A CN114138196A CN 114138196 A CN114138196 A CN 114138196A CN 202111422631 A CN202111422631 A CN 202111422631A CN 114138196 A CN114138196 A CN 114138196A
- Authority
- CN
- China
- Prior art keywords
- change information
- data change
- target
- data
- message queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Abstract
The present application relates to a power system data storage method, apparatus, computer device, storage medium and computer program product. The method comprises the following steps: acquiring data change information of a target service system in the power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue; acquiring current data change information from a distributed message queue; judging whether the generation sequence of the current data change information is behind the target data change information; if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into a target memory, and updating the target data change information; otherwise, discarding the current data change information. The method can ensure that the data change information is written into the target memory according to the generation sequence, thereby improving the accuracy and efficiency of data storage of the power system.
Description
Technical Field
The present application relates to the field of power system technologies, and in particular, to a method, an apparatus, a computer device, a storage medium, and a computer program product for storing data of a power system.
Background
With the rapid development of the power system technology, data information of each service system in the power system needs to be processed and analyzed, and data of each different service system in the power system is written into a target memory. At present, the data of the power system tends to change in characteristics such as large data volume, multiple data types, frequent change, high timeliness of real-time analysis and the like, when the data of each service system in the power system is stored, when the data change information is acquired, the data change information is sent to a distributed message queue, and the distributed message queue stores the data change information into a server of a corresponding distributed storage system; and reading the data change information successively through the distributed message queue and writing the data change information into the target memory. However, due to the influence of factors such as network reasons, the order of storing the data change information into the target storage is inconsistent with the order of storing the data change information into the servers in the distributed storage system, so that the writing order is disordered when the data change information is written into the target storage, and the accuracy and efficiency of data storage in the power system are reduced.
Therefore, how to improve the accuracy and efficiency of data storage of the power system is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
In view of the above, it is necessary to provide a power system data storage method, apparatus, computer device, computer readable storage medium and computer program product capable of improving accuracy and efficiency of data storage of a power system in view of the above technical problems.
In a first aspect, the present application provides a power system data storage method. The method comprises the following steps:
acquiring data change information of a target service system in an electric power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
acquiring current data change information from the distributed message queue;
judging whether the generation sequence of the current data change information is behind target data change information or not; wherein, the target data change information is the latest data change information written into the target memory;
if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into the target memory, and updating the target data change information;
otherwise, discarding the current data change information.
In one embodiment, the acquiring data change information of a target service system in an electric power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue includes:
acquiring the data change information of the target service system in the power system, and determining a data identifier corresponding to the data change information; wherein the data identifier indicates a generation order of the data variation information;
sending the data change information and a data identifier corresponding to the data change information to the distributed message queue;
and writing the data change information and the corresponding data identification into a server of the corresponding distributed storage system through the distributed message queue.
In one embodiment, the determining whether the generation order of the current data change information is after the target data change information includes:
acquiring a target data identifier corresponding to the target data change information from a cache;
judging whether the current data identifier of the current data change information is larger than the target data identifier;
if the current data identifier is larger than the target data identifier, indicating that the generation sequence of the current data change information is behind the target data change information;
and if the current data identifier is smaller than the target data identifier, indicating that the generation sequence of the current data change information is before the target data change information.
In one embodiment, the data change information in the cache is stored in the form of a key-value pair, where a key of the key-value pair includes a service system type, a table name, and a data key value corresponding to the data change information, and a value of the key-value pair includes a data identifier corresponding to the data change information;
the obtaining of the target data identifier corresponding to the target data change information from the cache includes:
determining target data change information corresponding to the current data change information from the cache according to the key of the key value pair;
and acquiring the target data identification corresponding to the target data change information.
In one embodiment, the sending the data change information and the data identifier corresponding to the data change information to the distributed message queue includes:
setting a plurality of distributed message queues, and determining the corresponding relation between each distributed message queue and the data type of the data change information;
determining a target data type of the data change information, and determining a corresponding target distributed message queue by using the target data type and the corresponding relation;
and sending the data change information and a data identifier corresponding to the data change information to the target distributed message queue.
In one embodiment, the writing, by the distributed message queue, the data change information and the corresponding data identifier into a server of a corresponding distributed storage system includes:
and writing the data change information and the corresponding data identification into the server of the corresponding distributed storage system in a polling mode through the distributed message queue.
In a second aspect, the present application further provides a power system data storage device. The device comprises:
the sending module is used for acquiring data change information of a target service system in the power system and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
the acquisition module is used for acquiring current data change information from the distributed message queue;
the judging module is used for judging whether the generation sequence of the current data change information is behind the target data change information or not; wherein, the target data change information is the latest data change information written into the target memory;
a first execution module, configured to write the current data change information into the target memory and update the target data change information if the generation sequence of the current data change information is after the target data change information;
a second executing module, configured to discard the current data change information if the generation order of the current data change information is before the target data change information.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the following steps when executing the computer program:
acquiring data change information of a target service system in an electric power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
acquiring current data change information from the distributed message queue;
judging whether the generation sequence of the current data change information is behind target data change information or not; wherein, the target data change information is the latest data change information written into the target memory;
if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into the target memory, and updating the target data change information;
otherwise, discarding the current data change information.
In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring data change information of a target service system in an electric power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
acquiring current data change information from the distributed message queue;
judging whether the generation sequence of the current data change information is behind target data change information or not; wherein, the target data change information is the latest data change information written into the target memory;
if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into the target memory, and updating the target data change information;
otherwise, discarding the current data change information.
In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which when executed by a processor performs the steps of:
acquiring data change information of a target service system in an electric power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
acquiring current data change information from the distributed message queue;
judging whether the generation sequence of the current data change information is behind target data change information or not; wherein, the target data change information is the latest data change information written into the target memory;
if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into the target memory, and updating the target data change information;
otherwise, discarding the current data change information.
The method comprises the steps of storing data change information into a server in a distributed storage system through a distributed message queue, judging whether the generation sequence of the current data change information is the target data change information which is newly written into a target memory or not after acquiring the current data change information from the distributed message queue and before storing the current data change information into the target memory, writing the current data change information into the target memory under the condition that the generation sequence of the current data change information is determined to be behind the target data change information, and discarding the disordered current data change information; therefore, the method can ensure that the data change information is written into the target memory according to the generation sequence, thereby improving the accuracy and efficiency of data storage of the power system.
Drawings
FIG. 1 is a diagram of an exemplary embodiment of a power system data storage method;
FIG. 2 is a schematic flow chart diagram of a method for storing data in a power system according to one embodiment;
FIG. 3 is a schematic flow chart of a data storage method for a power system according to another embodiment;
FIG. 4 is a block diagram of a power system data storage device in one embodiment;
FIG. 5 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The data storage method of the power system provided by the embodiment of the application can be applied to the application environment shown in fig. 1; the application environment comprises a power system 102 and a processing terminal 104, wherein the power system 102 is in communication connection with the processing terminal 104; the processing terminal 104 acquires data change information of a target service system of the power system 102, and writes the data change information into a server of a corresponding distributed storage system through a distributed message queue; and then acquiring current data change information from the distributed message queue, and writing the current data change information into a target memory according to whether the generation sequence of the current data change information is behind the target data change information or not so as to write the data change information into the target memory according to the generation sequence. The processing terminal 104 may be specifically a terminal device or a server; the terminal device may be, but not limited to, various personal computers, notebook computers, and power devices provided with processors, and the server may be implemented by an independent server or a server cluster formed by a plurality of servers.
In one embodiment, as shown in fig. 2, a power system data storage method is provided, which is described by taking the method as an example for being applied to the terminal device in fig. 1, and includes the following steps:
step 202, acquiring data change information of a target service system in the power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue.
It can be understood that, in the power system, corresponding service systems are respectively provided for different service requirements; the data change information is information which indicates that the corresponding data information changes due to operation in the service system, and the data information changes comprise changes of a database corresponding to the service system, tables in the database, addition, deletion, modification and the like of row and column values in the tables; for example, the bill of 10 months is newly added as a piece of data change information, and for example, a piece of data change information is correspondingly generated by deleting a device with the number of 1. Specifically, the log generally includes all details of the data change, so the present embodiment obtains the data change information corresponding to the target service system by analyzing the log of the relational database of the power system.
After the data change information is acquired, the data change information is converted into data information in a preset format and the data information is sent to a distributed message queue, after the data information in the preset format is acquired, the distributed message queue analyzes the data information in a corresponding mode to obtain corresponding data change information, and then the data change information is written into a server in a corresponding distributed storage system, namely the acquired data change information is dispersedly stored in a plurality of independent servers through the distributed storage system.
Step 204, obtaining current data change information from a distributed message queue;
step 206, judging whether the generation sequence of the current data change information is behind the target data change information; the target data change information is the latest data change information written into the target memory.
It should be noted that the target storage refers to a storage device that finally stores data change information; according to the property of the queue, current data change information is acquired from the distributed message queue according to the first-in first-out sequence, then the generation time of the target data change information which is newly written into the target memory is acquired, the generation time of the current data change information and the generation time of the target change information are compared, and whether the generation time of the current data change information is behind the generation time of the target data change information or not is judged, namely whether the generation sequence of the current data change information is behind the target data change information or not is judged.
Step 208, if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into the target memory, and updating the target data change information;
step 210, otherwise, discarding the current data change information.
Specifically, if it is determined that the generation order of the current data change information is after the target data change information, that is, the current data change information is changed data information generated in response to the re-execution of the operation after the target data change information, the current data change information is written into the target memory, and the data information stored in the target memory is updated using the current data change information. For example, assume that the data information currently stored in the target memory is a, the current data change information is a change from a to B, and the current data change information is written into the target memory, that is, the data information stored in the target memory is changed into B; the target data change information is then updated with the current data change information, i.e., the data change information that was most recently written to the target memory is updated.
In addition, if it is determined that the generation order of the current data change information is before the target data change information, it indicates that the current data change information is change data information generated in response to an operation performed before the target data change information, and indicates that the content of the current data change information is already overwritten, and therefore the current data change information is discarded.
It should be noted that, in actual operation, after the current data change information is written into the target memory, or after the current data change information is discarded, the process may continue to return to step 206, and obtain the current data change information from the distributed message queue to process the data change information in the distributed message queue.
The data storage method of the power system comprises the steps that data change information is stored in a server in a distributed storage system through a distributed message queue, after current data change information is obtained from the distributed message queue and before the current data change information is stored in a target memory, whether the generation sequence of the current data change information is the target data change information which is newly written into the target memory is judged, and then the current data change information is written into the target memory under the condition that the generation sequence of the current data change information is determined to be behind the target data change information, and disordered current data change information is discarded; therefore, the method can ensure that the data change information is written into the target memory according to the generation sequence, thereby improving the accuracy and efficiency of data storage of the power system.
On the basis of the foregoing embodiment, this embodiment further describes and optimizes the technical solution, and specifically, in this embodiment, acquiring data change information of a target service system in an electric power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue includes:
acquiring data change information of a target service system in the power system, and determining a data identifier corresponding to the data change information; wherein, the data identification represents the generation sequence of the data change information;
sending the data change information and a data identifier corresponding to the data change information to a distributed message queue;
and writing the data change information and the corresponding data identification into the server of the corresponding distributed storage system through the distributed message queue.
Specifically, in this embodiment, after data change information of a target service system in an electric power system is acquired, a data identifier corresponding to the data change information is further determined, where the data identifier is sequentially incremental data that is set corresponding to a generation order of the data change information, and each data identifier corresponds to each data change information one-to-one, that is, the corresponding data change information is represented by a data identifier, for example, a data identifier of data change information generated at a first time is 1, a data identifier of data change information generated at a next time is 2, and so on, a data identifier corresponding to each data change information is determined, and a generation order of each data change information is compared by the data identifier conveniently and intuitively.
Correspondingly, the data change information and the data identifier I corresponding to the data change information are sent to a distributed message queue, and after the data change information and the corresponding data identifier are received by the distributed message queue, the data change information and the corresponding data identifier are written into a server of a corresponding distributed storage system.
As a preferred embodiment, the determining whether the generation order of the current data change information is after the target data change information includes:
acquiring a target data identifier corresponding to the target data change information from the cache;
judging whether the current data identifier of the current data change information is larger than the target data identifier;
if the current data identifier is larger than the target data identifier, indicating that the generation sequence of the current data change information is behind the target data change information;
and if the current data identifier is smaller than the target data identifier, indicating that the generation sequence of the current data change information is before the target data change information.
Specifically, in actual operation, after the current data change information is written into the target memory, the target data change information is updated, and the updated target data change information and the corresponding target data identifier are stored in the cache; after acquiring the current data change information from the distributed message queue, determining a current data identifier corresponding to the current data change information, and acquiring a target data identifier corresponding to the target data change information from a cache; the data identification is generated along with the data change information generation sequence in an increasing mode, so that the larger the numerical value of the data identification is, the later the data change information generation sequence corresponding to the data identification is; therefore, the sizes of the current data identifier and the current data identifier are compared, and whether the current data identifier of the current data change information is larger than the target data identifier of the target data change information or not is judged; if the current data identifier is larger than the target data identifier, indicating that the generation sequence of the current data change information is behind the target data change information; and if the current data identifier is smaller than the target data identifier, indicating that the generation sequence of the current data change information is before the target data change information.
As can be seen, in the embodiment, the corresponding data identifier is set for each piece of data variation information, and the generation order of the data variation information is represented by the data identifier, so that the generation order of each piece of data variation information can be conveniently determined by comparing the sizes of the data identifiers.
On the basis of the foregoing embodiment, this embodiment further describes and optimizes the technical solution, and specifically, in this embodiment, the data change information in the cache is stored in the form of a key-value pair, where a key of the key-value pair includes a service system type, a table name, and a data key value corresponding to the data change information, and a value of the key-value pair includes a data identifier corresponding to the data change information;
acquiring a target data identifier corresponding to the target data change information from the cache, wherein the target data identifier comprises:
determining target data change information corresponding to the current data change information from the cache according to the key of the key value pair;
and acquiring a target data identifier corresponding to the target data change information.
In actual operation, data change information of a plurality of different service systems may be stored in the cache; in order to distinguish the service system and the data table corresponding to each data change information, the present embodiment stores the data change information in a form of a key-value pair, where a key of the key-value pair includes a service system type, a table name, and a data key value corresponding to the data change information, that is, a location where the data change information is modified correspondingly can be specifically determined according to information such as the service system name, the table name, and the data key value; the value of the key-value pair includes a data identification corresponding to the data variation information.
After acquiring the current data change information from the distributed message queue, analyzing the current data change information to obtain the current service system type, the current table name, the current data key value and the like corresponding to the current data change information; and then determining target data change information corresponding to the current data change information from the cache according to the analyzed content and the key of each key value pair in the cache, and further determining a target data identifier corresponding to the target data change information.
Therefore, in the embodiment, the data change information is stored in the cache in a key value pair mode, so that the target data change information corresponding to the current data change information can be conveniently and accurately determined, and the target data identifier is further determined.
On the basis of the foregoing embodiment, this embodiment further describes and optimizes the technical solution, and specifically, in this embodiment, sending the data change information and the data identifier corresponding to the data change information to the distributed message queue includes:
setting a plurality of distributed message queues, and determining the corresponding relation between each distributed message queue and the data type of the data change information;
determining a target data type of the data change information, and determining a corresponding target distributed message queue by using the target data type and the corresponding relation;
and sending the data change information and the data identification corresponding to the data change information to a target distributed message queue.
Specifically, according to the characteristics of a large data volume, a large number of data types, and the like of the data of the power system, a plurality of distributed message queues are set, and the corresponding relationship between each distributed message queue and the data type of the data change information is determined; the data type refers to a type corresponding to data change information, such as voltage data, current data, device information, and the like, and the specific content of the data type is not limited in this embodiment; a distributed message queue may correspond to one or more data types, which is not limited in this embodiment. For example, three distributed message queues are provided for storing voltage data, current data and device information, respectively; after acquiring the variable data information of the power system, determining a target data type of the variable data information, assuming the target data type is voltage data; determining a target distributed message queue corresponding to the target data type according to the corresponding relation between the distributed message queue and the data type, namely determining the target distributed message queue for storing the voltage data; and then sending the data change information and the data identification corresponding to the data change information to a target distributed message queue. More specifically, in actual operation, each distributed message queue corresponds to a distributed storage system, one distributed message queue may correspond to a preset number of servers designated in the distributed storage system, and the distributed message queue stores the received data change information in the corresponding server in the distributed storage system.
In the embodiment, the plurality of distributed message queues are arranged, and the data change information corresponding to the data type is stored by using each distributed message queue, so that the orderliness and reliability of the data storage process can be further ensured, and the condition of data disorder is avoided.
On the basis of the foregoing embodiment, this embodiment further describes and optimizes the technical solution, and specifically, in this embodiment, writing the data change information and the corresponding data identifier into the server of the corresponding distributed storage system through the distributed message queue includes:
and writing the data change information and the corresponding data identification into the server of the corresponding distributed storage system in a polling mode through the distributed message queue.
In this embodiment, after receiving the data change information and the corresponding data identifier each time, the distributed message queue alternately writes the data change information and the corresponding data identifier into servers in the distributed storage system in a Round Robin (RR) manner according to a polling sequence, so as to store each piece of data change information and the corresponding data identifier in each server in a balanced manner. More specifically, each distributed message queue may designate a plurality of servers, and after receiving the data change information and the corresponding data identifier, the distributed message queue writes the data change information and the corresponding data identifier into the designated plurality of servers of the corresponding distributed storage system in a polling manner.
Therefore, in the embodiment, the data change information and the corresponding data identifier are written into the corresponding server of the distributed storage system in a polling manner, so that the storage capacity of each server in the distributed storage system can be balanced, the efficiency of the distributed storage system is improved, and the data skew is reduced.
In order to make those skilled in the art better understand the technical solutions in the present application, the following describes the technical solutions in the embodiments of the present application in detail with reference to practical application scenarios. In the embodiment of the application, a method for storing data of a power system comprises the following specific steps:
step 302, acquiring data change information of a target service system in the power system, and determining a data identifier corresponding to the data change information: adding power equipment in a power system, and adding information in a database of a service system corresponding to the power equipment: the device id is 1, the device name is 1, and the data identifier pos corresponding to the data change information is 1; after 5 seconds, modifying the information of the power equipment, wherein the equipment id is 1, the equipment name is 2, and the data identification pos corresponding to the data change information is 2; after 1 minute, the power equipment is removed from the power system, the equipment information of the power equipment is correspondingly deleted, and the data identification pos corresponding to the data change information is 3.
Step 304, determining target data types corresponding to the data change information respectively, and determining distributed message queues corresponding to the data change information respectively according to the corresponding relation between the distributed message queues and the data types of the data change information;
step 306, sending each data change information and the data identifier corresponding to the data change information to the corresponding distributed message queue;
step 308, each distributed message queue writes the data change information and the corresponding data identifier into a server of the corresponding distributed storage system in a polling manner; in the distributed storage system, a server 1 stores partial information of a distributed message queue 1 and partial information of a distributed message queue 2; the server 2 stores part of information of the distributed message queue 1 and part of information of the distributed message queue 3; the server 3 stores part of the information of the distributed message queue 2 and part of the information of the distributed message queue 3. In addition, in actual operation, the data change information stored in the distributed message queue is cleared according to a preset time period.
Step 310, obtaining current data change information from a distributed message queue, and determining a current data identifier corresponding to the current data change information;
step 312, according to the information of the service system, the table, the data key value and the like of the current data change information, determining the target data change information corresponding to the current data change information from the cache according to the key of the key value pair, and acquiring the target data identifier corresponding to the target data change information;
step 314, judging whether the current data identifier of the current data change information is larger than the target data identifier;
step 316, if the current data identifier is larger than the target data identifier, indicating that the generation sequence of the current data change information is behind the target data change information, writing the current data change information into the target memory, and updating the target data change information;
step 318, if the current data flag is smaller than the target data flag, indicating that the generation sequence of the current data change information is before the target data change information, discarding the current data change information.
The data storage method of the power system comprises the steps that data change information is stored in a server in a distributed storage system through a distributed message queue, after current data change information is obtained from the distributed message queue and before the current data change information is stored in a target memory, whether the generation sequence of the current data change information is the target data change information which is newly written into the target memory is judged, and then the current data change information is written into the target memory under the condition that the generation sequence of the current data change information is determined to be behind the target data change information, and disordered current data change information is discarded; therefore, the method can ensure that the data change information is written into the target memory according to the generation sequence, thereby improving the accuracy and efficiency of data storage of the power system.
It should be understood that, although the steps in the flowcharts related to the embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
Based on the same inventive concept, the embodiment of the present application further provides a power system data storage device for implementing the above-mentioned power system data storage method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so specific limitations in one or more embodiments of the power system data storage device provided below can be referred to the limitations of the power system data storage method in the above, and are not described herein again.
In one embodiment, as shown in fig. 4, there is provided a power system data storage device comprising: a sending module 402, an obtaining module 404, a judging module 406, a first executing module 408 and a second executing module 410, wherein:
a sending module 402, configured to obtain data change information of a target service system in an electric power system, and write the data change information into a server of a corresponding distributed storage system through a distributed message queue;
an obtaining module 404, configured to obtain current data change information from a distributed message queue;
a determining module 406, configured to determine whether a generation sequence of the current data change information is behind the target data change information; wherein, the target data change information is the latest data change information written into the target memory;
a first executing module 408, configured to write the current data change information into the target memory and update the target data change information if the generation sequence of the current data change information is after the target data change information;
the second executing module 410 is configured to discard the current data change information if the generation order of the current data change information is before the target data change information.
The power system data storage device provided by the embodiment of the invention has the same beneficial effects as the power system data storage method.
In one embodiment, the sending module comprises:
the identification determining submodule is used for acquiring data change information of a target service system in the power system and determining a data identification corresponding to the data change information; wherein, the data identification represents the generation sequence of the data change information;
the sending submodule is used for sending the data change information and the data identification corresponding to the data change information to the distributed message queue;
and the storage submodule is used for writing the data change information and the corresponding data identification into the server of the corresponding distributed storage system through the distributed message queue.
In one embodiment, the determining module comprises:
the acquisition submodule is used for acquiring a target data identifier corresponding to the target data change information from the cache;
the judgment submodule is used for judging whether the current data identifier of the current data change information is larger than the target data identifier; if the current data identifier is larger than the target data identifier, indicating that the generation sequence of the current data change information is behind the target data change information; and if the current data identifier is smaller than the target data identifier, indicating that the generation sequence of the current data change information is before the target data change information.
In one embodiment, the acquisition submodule includes:
the data determining submodule is used for determining target data change information corresponding to the current data change information from the cache according to the key of the key value pair; the data change information in the cache is stored in a key-value pair mode, the key of the key-value pair comprises a business system type, a table name and a data key value corresponding to the data change information, and the value of the key-value pair comprises a data identifier corresponding to the data change information;
and the identifier obtaining submodule is used for obtaining the target data identifier corresponding to the target data change information.
In one embodiment, the transmitting submodule includes:
the setting unit is used for setting a plurality of distributed message queues and determining the corresponding relation between each distributed message queue and the data type of the data change information;
the determining unit is used for determining a target data type of the data change information and determining a corresponding target distributed message queue by using the target data type and the corresponding relation;
and the sending unit is used for sending the data change information and the data identification corresponding to the data change information to the target distributed message queue.
In one embodiment, the storage submodule includes:
and the storage unit is used for writing the data change information and the corresponding data identification into the server of the corresponding distributed storage system in a polling mode through the distributed message queue.
The modules in the power system data storage device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 5. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a power system data storage method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 5 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring data change information of a target service system in the power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
acquiring current data change information from a distributed message queue;
judging whether the generation sequence of the current data change information is behind the target data change information; wherein, the target data change information is the latest data change information written into the target memory;
if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into a target memory, and updating the target data change information;
otherwise, discarding the current data change information.
The computer equipment provided by the embodiment of the invention has the same beneficial effects as the power system data storage method.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring data change information of a target service system in the power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
acquiring current data change information from a distributed message queue;
judging whether the generation sequence of the current data change information is behind the target data change information; wherein, the target data change information is the latest data change information written into the target memory;
if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into a target memory, and updating the target data change information;
otherwise, discarding the current data change information.
The computer-readable storage medium provided by the embodiment of the invention has the same beneficial effects as the above power system data storage method.
In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, performs the steps of:
acquiring data change information of a target service system in the power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
acquiring current data change information from a distributed message queue;
judging whether the generation sequence of the current data change information is behind the target data change information; wherein, the target data change information is the latest data change information written into the target memory;
if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into a target memory, and updating the target data change information;
otherwise, discarding the current data change information.
The computer program product provided by the embodiment of the invention has the same beneficial effects as the above power system data storage method.
It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (10)
1. A power system data storage method, comprising:
acquiring data change information of a target service system in an electric power system, and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
acquiring current data change information from the distributed message queue;
judging whether the generation sequence of the current data change information is behind target data change information or not; wherein, the target data change information is the latest data change information written into the target memory;
if the generation sequence of the current data change information is behind the target data change information, writing the current data change information into the target memory, and updating the target data change information;
otherwise, discarding the current data change information.
2. The method of claim 1, wherein the obtaining data change information of a target service system in the power system and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue comprises:
acquiring the data change information of the target service system in the power system, and determining a data identifier corresponding to the data change information; wherein the data identifier indicates a generation order of the data variation information;
sending the data change information and a data identifier corresponding to the data change information to the distributed message queue;
and writing the data change information and the corresponding data identification into a server of the corresponding distributed storage system through the distributed message queue.
3. The method of claim 2, wherein said determining whether the current data change information is generated in an order subsequent to the target data change information comprises:
acquiring a target data identifier corresponding to the target data change information from a cache;
judging whether the current data identifier of the current data change information is larger than the target data identifier;
if the current data identifier is larger than the target data identifier, indicating that the generation sequence of the current data change information is behind the target data change information;
and if the current data identifier is smaller than the target data identifier, indicating that the generation sequence of the current data change information is before the target data change information.
4. The method of claim 3, wherein the data change information in the cache is stored in a form of a key-value pair, a key of the key-value pair comprises a service system type, a table name and a data key value corresponding to the data change information, and a value of the key-value pair comprises a data identifier corresponding to the data change information;
the obtaining of the target data identifier corresponding to the target data change information from the cache includes:
determining target data change information corresponding to the current data change information from the cache according to the key of the key value pair;
and acquiring the target data identification corresponding to the target data change information.
5. The method of claim 2, wherein sending the data change information and the data identifier corresponding to the data change information to the distributed message queue comprises:
setting a plurality of distributed message queues, and determining the corresponding relation between each distributed message queue and the data type of the data change information;
determining a target data type of the data change information, and determining a corresponding target distributed message queue by using the target data type and the corresponding relation;
and sending the data change information and a data identifier corresponding to the data change information to the target distributed message queue.
6. The method according to any one of claims 2 to 5, wherein the writing the data change information and the corresponding data identifier into the server of the corresponding distributed storage system through the distributed message queue comprises:
and writing the data change information and the corresponding data identification into the server of the corresponding distributed storage system in a polling mode through the distributed message queue.
7. An electrical power system data storage apparatus, the apparatus comprising:
the sending module is used for acquiring data change information of a target service system in the power system and writing the data change information into a server of a corresponding distributed storage system through a distributed message queue;
the acquisition module is used for acquiring current data change information from the distributed message queue;
the judging module is used for judging whether the generation sequence of the current data change information is behind the target data change information or not; wherein, the target data change information is the latest data change information written into the target memory;
a first execution module, configured to write the current data change information into the target memory and update the target data change information if the generation sequence of the current data change information is after the target data change information;
a second executing module, configured to discard the current data change information if the generation order of the current data change information is before the target data change information.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 6.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111422631.9A CN114138196A (en) | 2021-11-26 | 2021-11-26 | Power system data storage method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111422631.9A CN114138196A (en) | 2021-11-26 | 2021-11-26 | Power system data storage method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114138196A true CN114138196A (en) | 2022-03-04 |
Family
ID=80388334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111422631.9A Pending CN114138196A (en) | 2021-11-26 | 2021-11-26 | Power system data storage method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114138196A (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101416505A (en) * | 2006-03-28 | 2009-04-22 | 株式会社东芝 | Method of processing video data and wireless communication apparatus |
| CN103024014A (en) * | 2012-12-03 | 2013-04-03 | 北京京东世纪贸易有限公司 | Method and system for distributing and processing mass data passing through message queues |
| CN105243812A (en) * | 2015-10-23 | 2016-01-13 | 北京中科汉天下电子技术有限公司 | Method and system for forwarding data package |
| CN105760236A (en) * | 2016-05-13 | 2016-07-13 | 中国工商银行股份有限公司 | Data collection method and system of distributed computer cluster |
| CN106878365A (en) * | 2016-06-08 | 2017-06-20 | 阿里巴巴集团控股有限公司 | A kind of method of data synchronization and equipment |
| CN107609085A (en) * | 2017-09-07 | 2018-01-19 | 国网辽宁省电力有限公司 | A kind of intelligent grid metric data processing method and system based on big data technology |
| CN108062368A (en) * | 2017-12-08 | 2018-05-22 | 北京百度网讯科技有限公司 | Full dose data translating method, device, server and storage medium |
| CN110647409A (en) * | 2019-09-23 | 2020-01-03 | 深圳市网心科技有限公司 | Message writing method, electronic device, system and medium |
| CN110765091A (en) * | 2019-09-09 | 2020-02-07 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Account checking method and system |
| CN111680046A (en) * | 2020-08-12 | 2020-09-18 | 国网浙江省电力有限公司营销服务中心 | Fast storage method for high-frequency power data |
| KR20200116775A (en) * | 2019-04-02 | 2020-10-13 | 엘에스일렉트릭(주) | Database, database management system and management method |
| CN111988674A (en) * | 2020-08-18 | 2020-11-24 | 广州极飞科技有限公司 | Multimedia data transmission method, device, equipment and storage medium |
| CN113076304A (en) * | 2021-04-16 | 2021-07-06 | 北京沃东天骏信息技术有限公司 | Distributed version management method, device and system |
-
2021
- 2021-11-26 CN CN202111422631.9A patent/CN114138196A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101416505A (en) * | 2006-03-28 | 2009-04-22 | 株式会社东芝 | Method of processing video data and wireless communication apparatus |
| CN103024014A (en) * | 2012-12-03 | 2013-04-03 | 北京京东世纪贸易有限公司 | Method and system for distributing and processing mass data passing through message queues |
| CN105243812A (en) * | 2015-10-23 | 2016-01-13 | 北京中科汉天下电子技术有限公司 | Method and system for forwarding data package |
| CN105760236A (en) * | 2016-05-13 | 2016-07-13 | 中国工商银行股份有限公司 | Data collection method and system of distributed computer cluster |
| CN106878365A (en) * | 2016-06-08 | 2017-06-20 | 阿里巴巴集团控股有限公司 | A kind of method of data synchronization and equipment |
| CN107609085A (en) * | 2017-09-07 | 2018-01-19 | 国网辽宁省电力有限公司 | A kind of intelligent grid metric data processing method and system based on big data technology |
| CN108062368A (en) * | 2017-12-08 | 2018-05-22 | 北京百度网讯科技有限公司 | Full dose data translating method, device, server and storage medium |
| KR20200116775A (en) * | 2019-04-02 | 2020-10-13 | 엘에스일렉트릭(주) | Database, database management system and management method |
| CN110765091A (en) * | 2019-09-09 | 2020-02-07 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Account checking method and system |
| CN110647409A (en) * | 2019-09-23 | 2020-01-03 | 深圳市网心科技有限公司 | Message writing method, electronic device, system and medium |
| CN111680046A (en) * | 2020-08-12 | 2020-09-18 | 国网浙江省电力有限公司营销服务中心 | Fast storage method for high-frequency power data |
| CN111988674A (en) * | 2020-08-18 | 2020-11-24 | 广州极飞科技有限公司 | Multimedia data transmission method, device, equipment and storage medium |
| CN113076304A (en) * | 2021-04-16 | 2021-07-06 | 北京沃东天骏信息技术有限公司 | Distributed version management method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114089921A (en) | Power system data storage method and device, computer equipment and storage medium | |
| CN116991800A (en) | File acquisition system, method, device, computer equipment and storage medium | |
| CN113111078B (en) | Resource data processing method and device, computer equipment and storage medium | |
| CN114138196A (en) | Power system data storage method and device, computer equipment and storage medium | |
| CN112035471A (en) | Transaction processing method and computer equipment | |
| CN117056220A (en) | Batch job testing method and device, computer equipment and storage medium | |
| CN116910115A (en) | Group query method, device, computer equipment and storage medium | |
| CN116541145A (en) | Data calling method, device, computer equipment and computer storage medium | |
| CN114647630A (en) | File synchronization method, information generation method, file synchronization device, information generation device, computer equipment and storage medium | |
| CN114092064A (en) | Electric power big data processing batch processing system, method, device, equipment and medium | |
| CN117390011A (en) | Report data processing method, device, computer equipment and storage medium | |
| CN116708245A (en) | Interface control method, device and computer equipment | |
| CN117951223A (en) | Data synchronization method, device, computer equipment and storage medium | |
| CN117764529A (en) | Target item determining method, device, computer equipment, storage medium and product | |
| CN115510130A (en) | Report generation method and device, computer equipment and storage medium | |
| CN116738000A (en) | Data storage relationship processing method and device, electronic equipment and storage medium | |
| CN117435589A (en) | Data transfer method, device, computer equipment and storage medium | |
| CN114925087A (en) | Data acquisition method and device, computer equipment and storage medium | |
| CN117349320A (en) | Data processing method, device, computer equipment and storage medium | |
| CN116204541A (en) | Multidimensional database updating method, multidimensional database updating device, computer equipment and storage medium | |
| CN116866419A (en) | Information pushing method, device, computer equipment and storage medium | |
| CN114519062A (en) | Basic data updating method and device, computer equipment and storage medium | |
| CN116521546A (en) | Interface performance adjusting method and device, computer equipment and storage medium | |
| CN114064494A (en) | Data abnormity alarm method and device, electronic equipment and computer readable medium | |
| CN117234562A (en) | Configuration parameter updating method and device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |