CN108805741B - Fusion method, device and system of power quality data - Google Patents
Fusion method, device and system of power quality data Download PDFInfo
- Publication number
- CN108805741B CN108805741B CN201810368579.5A CN201810368579A CN108805741B CN 108805741 B CN108805741 B CN 108805741B CN 201810368579 A CN201810368579 A CN 201810368579A CN 108805741 B CN108805741 B CN 108805741B
- Authority
- CN
- China
- Prior art keywords
- power quality
- quality data
- data
- offset
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/06—Electricity, gas or water supply
Abstract
The method for fusing the power quality data comprises the following steps: the method comprises the steps that power quality data are collected by power quality monitoring points and sent to a kafka server in real time according to subscription instructions; the Kafka server receives a data consumption request of a data WEB client; and the kafka server searches the offset corresponding to the data consumption request of the WEB client according to the data consumption record of the WEB client, acquires the electric energy quality data corresponding to the offset from the kafka server and sends the electric energy quality data to the WEB client. The kafka server is used for transferring data, so that the power quality data of different systems can be effectively fused, and the power quality data request of the WEB client can be responded in real time by responding to the offset through the kafka server.
Description
Technical Field
The invention belongs to the field of electric energy quality data processing, and particularly relates to a method, a device and a system for fusing electric energy quality data.
Background
With the continuous development of the smart power grid and the increasing attention paid to the power quality by people, the mass of the current power quality data is increased. Due to the rapid growth of power quality data, power quality monitoring systems that monitor the power quality data are also increasing.
Since different monitoring systems usually employ different databases to store data, the different systems have significant differences in operating environments (e.g., computer operating systems, database systems, information security partitions, etc.), network structures and protocols, data models, physical models, data semantics, and the like. Therefore, for massive power quality data, management and transmission among different monitoring systems cannot be effectively performed, and the difficulty of power monitoring data fusion is high.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, and a system for fusing power quality data, so as to solve the problems in the prior art that, for massive power quality data, management and transmission cannot be effectively performed between different monitoring systems, and the difficulty of fusing power monitoring data is high.
The first aspect of the embodiments of the present invention provides a method for fusing power quality data, where the method for fusing power quality data includes:
the method comprises the steps that power quality data are collected by power quality monitoring points and sent to a kafka server in real time according to subscription instructions;
the Kafka server receives a data consumption request of a data WEB client;
and the kafka server searches the offset corresponding to the data consumption request of the WEB client according to the data consumption record of the WEB client, acquires the electric energy quality data corresponding to the offset from the kafka server and sends the electric energy quality data to the WEB client.
With reference to the first aspect, in a first possible implementation manner of the first aspect, the step of acquiring power quality data by the power quality monitoring terminal, and sending the power quality data to the kafka server in real time according to a subscription instruction includes:
collecting power quality data in real time through power quality monitoring points;
receiving a subscription instruction sent by a kafka server through a WEB service interface;
and encapsulating the power quality data into a message and sending the message to a kafka server in real time, wherein the kafka server receives the message and decapsulates the message to obtain the power quality data, and determines the offset of the received power quality data.
With reference to the first aspect, in a second possible implementation manner of the first aspect, the determining an offset of the received power quality data includes:
transmitting the power quality data to a specified subject and a partition;
determining an offset of the power quality data based on an offset of the end of the specified topic and the partition.
With reference to the first aspect, the first possible implementation manner of the first aspect, and the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the method further includes:
performing redundant backup on the power quality data and the corresponding offset in a kafka server cluster mode;
and determining the kafka server in the kafka server cluster to provide the service for the WEB client according to the service state of the partition in the kafka server.
With reference to the first aspect, in a fourth possible implementation manner of the first aspect, the method further includes:
and the kafka server deletes the electric energy quality data stored earlier according to a preset caching strategy and the storage sequence of the electric energy quality data.
A second aspect of the embodiments of the present invention provides a device for fusing power quality data, where the device for fusing power quality data includes:
the subscription data sending unit is used for acquiring power quality data by the power quality monitoring point and sending the power quality data to the kafka server in real time according to a subscription instruction;
the consumption request receiving unit is used for receiving a data consumption request of the data WEB client by the Kafka server;
and the offset data sending unit is used for searching the offset corresponding to the data consumption request of the WEB client by the kafka server according to the data consumption record of the WEB client, acquiring the electric energy quality data corresponding to the offset from the kafka server and sending the electric energy quality data to the WEB client.
With reference to the second aspect, in a first possible implementation manner of the second aspect, the subscription data sending unit includes:
the acquisition subunit is used for acquiring the power quality data in real time through the power quality monitoring points;
the instruction sending subunit is used for receiving a subscription instruction sent by the kafka server through a WEB service interface;
and the subscription message sending subunit is used for packaging the power quality data into a message and sending the message to the kafka server in real time, and the kafka server receives the message, decapsulates the message to obtain the power quality data and determines the offset of the received power quality data.
With reference to the second aspect, in a second possible implementation manner of the second aspect, the subscription message sending subunit includes:
the partition determining module is used for sending the power quality data to a specified theme and a partition;
and the offset determining module is used for determining the offset of the power quality data on the basis of the offset of the specified theme and the tail of the partition.
A third aspect of the embodiments of the present invention provides a fusion system of power quality data, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the fusion method of power quality data according to any one of the first aspect when executing the computer program.
A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for fusing power quality data according to any one of the first aspect.
Compared with the prior art, the embodiment of the invention has the following beneficial effects: the power quality data collected by the power quality monitoring points are sent to the kafka server in real time according to the subscription instruction, when the kafka server receives a data consumption request sent by a WEB client, the offset corresponding to the WEB client is searched according to the consumption record of the WEB client, and the power quality data corresponding to the offset is sent to the WEB client. The kafka server is used for transferring data, so that the power quality data of different systems can be effectively fused, and the power quality data request of the WEB client can be responded in real time by responding to the offset through the kafka server.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a schematic diagram of a fusion system for power quality data provided by an embodiment of the invention;
fig. 2 is a schematic flow chart illustrating an implementation of a fusion method of power quality data according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an apparatus provided by an embodiment of the present invention;
fig. 4 is a schematic diagram of a fusion system of power quality data provided by an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Fig. 1 is a schematic diagram of a power quality data fusion system provided in an embodiment of the present application, where the power quality data fusion system includes a kafka (kaffka) server, a WEB client, and a power quality monitoring point, where:
the kafka server can be a plurality of servers, and a server cluster is formed by the plurality of servers. The plurality of servers may include a primary server and a backup server. Wherein, each server can be divided into a plurality of categories and partitions, and for the partitions in each category, different WEB clients can be provided with access services. The primary server and backup server may be configured according to the category and the content in the partition. For example, for a partition Y1 in category X1, server A is the primary server and server B is the backup server, while for partition Y2 in category X2, server B is the primary server and server A is the backup server. For the same power quality data, the power quality data can be synchronously backed up in the server cluster, and when the WEB client accesses the power quality data, the load of each server can be automatically balanced according to the service state of the server, so that the faster access speed can be conveniently provided for the WEB client.
The kafka server is a server of an open source stream processing platform developed by the Apache software foundation. The Kafka platform is a high-throughput distributed publish-subscribe messaging system that can handle all the action flow data in a consumer-scale website.
The power quality data are collected by power quality monitoring points and then sent to a kafka server, and the kafka server stores the power quality data in a specified category and a specified partition and provides corresponding offset for the power quality data. And through the offset, the WEB client can access any data. Certainly, the kafka server may also record the access or consumption record of the WEB client, so that the data to be sent is directly acquired when the WEB client accesses the WEB client.
The kafka server can acquire the electric energy quality data acquired by each electric energy quality monitoring point in a message subscription mode. When the kafka server sends a subscription instruction to the power quality monitoring point through a WEB service interface, the power quality data is packaged into a message by the power monitoring point and sent to the kafka server, and the kafka server can receive the power quality data collected by the power quality monitoring point in real time. When the kafka server cancels the subscription, the power quality data collected by the appointed power quality monitoring point cannot be received and synchronized to each server in the kafka server cluster, and when a user accesses, the accessed records are synchronized to each server.
The WEB client is a terminal capable of checking monitoring data and can be a smart phone, a tablet computer, a desktop computer and the like. The WEB client can send a request to the kafka server in a data viewing request sending mode. And when the kafka server receives the access request, searching corresponding access data according to the recorded offset corresponding to the WEB client.
Of course, the request may also carry an offset corresponding to the data to be accessed, and the kafka server may search for the corresponding data according to the offset and send the data to the WEB client.
Or, when the WEB client accesses the data for the first time, the offset data may be sent to the WEB client by the kafka server, so that the WEB client can search for the offset corresponding to the required data.
The power quality monitoring point can directly acquire data to be monitored and store the acquired data in a local system, or can package the power quality data into a message according to a subscription instruction, transmit the message to a kafka server in real time, and decapsulate the message by the kafka server to obtain the power quality data.
Fig. 2 is a schematic view of an implementation flow of a method for fusing power quality data provided in an embodiment of the present application, which is detailed as follows:
in step S201, the power quality monitoring point collects power quality data, and sends the power quality data to the kafka server in real time according to a subscription instruction;
specifically, the power quality monitoring point can acquire power quality data in real time, and for the acquired power quality data, the acquired power quality data can be stored at the power quality monitoring point, or the acquired power quality data can be packaged into a message in real time according to a subscription instruction transmitted by a WEB service interface, and the message is returned to the kafka server for decapsulation.
The power monitoring point is provided with a WEB service subscription interface and can receive a subscription instruction sent by the kafka server. When a subscription instruction is received, the electric energy monitoring point can send the electric energy monitoring data to the kafka server in real time, and when the subscription is cancelled, the real-time electric energy quality data is stopped to be published.
Preferably, the step of acquiring the power quality data by the power quality monitoring terminal and sending the power quality data to the kafka server in real time according to the subscription instruction may include:
a1, collecting power quality data in real time through power quality monitoring points;
a2, receiving a subscription instruction sent by a kafka server through a WEB service interface;
a3, encapsulating the power quality data into a message and sending the message to a kafka server in real time, wherein the kafka server receives the message and decapsulates the message to obtain the power quality data, and determines the offset of the received power quality data.
The power quality data may be stored in a designated class and a designated partition of the kafka server and an offset for each power quality data may be determined. The offset is determined according to the sequence of the power quality data, namely the offset of the prior power quality data is smaller, and the offset of the subsequent power quality data is larger. When the power quality data is sent, a proper serialization method can be selected, message contents are coded, batch message sending is realized by modifying configuration parameters, the coding can transmit the power quality data in a quadruple mode, and in order to avoid calculation and identification errors, all elements are represented as literal values in an operating environment in the coding process.
For example, the offset of the previous power quality data is n, when new power quality data is received, the offset accumulates numbers on the previous basis, the newly received power quality data is numbered from n +1, and the offset of the new power quality data is determined.
In step S202, the Kafka server receives a data consumption request from the data WEB client;
the kafka server can receive a consumption request of a WEB client for sending a data demand through an API (application programming interface).
For a new WEB client, because the power quality data is not obtained before, no access record is available on the kafka server, and the kafka server does not record the access history information of the kafka server, so that a corresponding table of the power quality data and the offset can be sent to the WEB client, the WEB client can send the offset corresponding to the data required to be requested to the kafka server, and the kafka server searches the corresponding data according to the offset sent by the WEB client and sends the corresponding data to the WEB client.
Of course, for new WEB clients, the kafka server may also transmit power quality data from the beginning.
The kafka server can comprise different types and partitions, and generally provides data transmission service for only one WEB client at the same time for the same partition, so that the WEB client can be effectively ensured to be capable of accessing the power quality data in real time and efficiently. After the WEB client finishes accessing, the access record may be recorded by the kafka server, and synchronized to the kafka server cluster, so as to facilitate the next access of the WEB client. Alternatively, the WEB client may record its own access record, and transmit the access record to the kafka server by a request at the time of the next access.
In step S203, the kafka server searches for an offset corresponding to the data consumption request of the WEB client according to the data consumption record of the WEB client, acquires the power quality data corresponding to the offset from the kafka server, and sends the power quality data to the WEB client.
And the kafka server searches the power quality data according to the searched offset, and when corresponding data is searched, a better resource configuration mode can be searched in the kafka server cluster according to a resource configuration strategy so as to transmit the required data to the WEB client.
The kafka server can comprise a main server and a backup server, and the power quality data and the corresponding offset can be subjected to redundant backup in a kafka server cluster mode; and determining the kafka server in the kafka server cluster to provide the service for the WEB client according to the service state of the partition in the kafka server.
The same power quality data and the offset corresponding to the power pinch data are backed up on different servers, when the kafka server receives a data demand, the corresponding category and the corresponding partition can be searched according to the offset, whether the partition provides the data transmission service or not is judged, and the efficient data transmission service is provided for the WEB client through the category and the partition in the idle state.
In addition, the main server and the backup server can be distinguished according to different data, for example, for data X, the server a is the main server, the server B is the backup server, for data B, the server B is the main server, and the server a is the backup server, so that access blockage of a certain server caused by data access by a WEB client can be effectively avoided, and the efficiency of data access is further improved.
In addition, in order to ensure that the kafka server can work normally and orderly, a data caching strategy can be preset, and newly collected data are not deleted immediately even after being sent to the WEB client and are stored in the kafka server for a period of time. When the deletion condition of the preset caching strategy is met, a part of earlier data stored in the kafka server is deleted, so that the aim of releasing the disk space is fulfilled, and the kafka server can be continuously used.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Fig. 3 is a schematic structural diagram of a device for fusing power quality data according to an embodiment of the present application, where the device for fusing power quality data includes:
the subscription data sending unit 301 is used for acquiring power quality data by the power quality monitoring point and sending the power quality data to the kafka server in real time according to a subscription instruction;
a consumption request receiving unit 302, configured to receive, by the Kafka server, a data consumption request of the data WEB client;
and the offset data sending unit 303 is configured to search, by the kafka server, an offset corresponding to the data consumption request of the WEB client according to the data consumption record of the WEB client, obtain, from the kafka server, the power quality data corresponding to the offset, and send the power quality data to the WEB client.
Preferably, the subscription data sending unit includes:
the acquisition subunit is used for acquiring the power quality data in real time through the power quality monitoring points;
the instruction sending subunit is used for receiving a subscription instruction sent by the kafka server through a WEB service interface;
and the subscription message sending subunit is used for packaging the power quality data into a message and sending the message to the kafka server in real time, and the kafka server receives the message, decapsulates the message to obtain the power quality data and determines the offset of the received power quality data.
Preferably, the subscription message sending subunit includes:
the partition determining module is used for sending the power quality data to a specified theme and a partition;
and the offset determining module is used for determining the offset of the power quality data on the basis of the offset of the specified theme and the tail of the partition.
The fusion device of the power quality data corresponds to the power quality data fusion method shown in fig. 2.
Fig. 4 is a schematic diagram of a fusion system of power quality data according to an embodiment of the present invention. As shown in fig. 4, the fusion system 4 of the power quality data of the embodiment includes: a processor 40, a memory 41 and a computer program 42 stored in said memory 41 and executable on said processor 40, such as a fusion program of power quality data. The processor 40 executes the computer program 42 to implement the steps in the above-mentioned embodiment of the method for fusing power quality data, such as the steps 201 to 203 shown in fig. 2. Alternatively, the processor 40, when executing the computer program 42, implements the functions of each module/unit in each device embodiment described above, for example, the functions of the modules 301 to 303 shown in fig. 3.
Illustratively, the computer program 42 may be partitioned into one or more modules/units that are stored in the memory 41 and executed by the processor 40 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions for describing the execution process of the computer program 42 in the fusion system 4 of power quality data. For example, the computer program 42 may be divided into a subscription data sending unit, a data request receiving unit, and an offset data sending unit, and each unit specifically functions as follows:
the subscription data sending unit is used for acquiring power quality data by the power quality monitoring point and sending the power quality data to the kafka server in real time according to a subscription instruction;
the consumption request receiving unit is used for receiving a data consumption request of the data WEB client by the Kafka server;
and the offset data sending unit is used for searching the offset corresponding to the data consumption request of the WEB client by the kafka server according to the data consumption record of the WEB client, acquiring the electric energy quality data corresponding to the offset from the kafka server and sending the electric energy quality data to the WEB client.
The fusion system of the power quality data may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that fig. 4 is merely an example of the power quality data fusion system 4, and does not constitute a limitation of the power quality data fusion system 4, and may include more or less components than those shown, or combine certain components, or different components, for example, the power quality data fusion system may further include input and output devices, network access devices, buses, etc.
The Processor 40 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 41 may be an internal storage unit of the power quality data fusion system 4, such as a hard disk or a memory of the power quality data fusion system 4. The memory 41 may also be an external storage device of the power quality data fusion system 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are equipped on the power quality data fusion system 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the fusion system 4 of the power quality data. The memory 41 is used for storing the computer program and other programs and data required by the fusion system of the power quality data. The memory 41 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. . Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media which may not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A method for fusing power quality data is characterized by comprising the following steps:
the method comprises the steps that power quality data are collected by power quality monitoring points and sent to a kafka server in real time according to subscription instructions;
the Kafka server receives a data consumption request of a data WEB client;
the kafka server searches an offset corresponding to a data consumption request of the WEB client according to the data consumption record of the WEB client, acquires power quality data corresponding to the offset from the kafka server, and sends the power quality data to the WEB client;
the kafka server comprises a server cluster formed by a plurality of servers, the plurality of servers comprise a main server and a standby server, the same power quality data and offset corresponding to the power quality data are backed up on different servers, when the kafka server receives a data demand, a corresponding type and a corresponding partition are searched according to the offset, whether the partition is providing data transmission service or not is judged, and the data transmission service is provided for the WEB client through the type and the partition in an idle state.
2. The method for fusing the power quality data according to claim 1, wherein the step of collecting the power quality data by the power quality monitoring point and sending the power quality data to the kafka server in real time according to the subscription instruction comprises the steps of:
collecting power quality data in real time through power quality monitoring points;
receiving a subscription instruction sent by a kafka server through a WEB service interface;
and encapsulating the power quality data into a message and sending the message to a kafka server in real time, wherein the kafka server receives the message and decapsulates the message to obtain the power quality data, and determines the offset of the received power quality data.
3. The method of fusing power quality data according to claim 2, wherein the step of determining an offset of the received power quality data comprises:
transmitting the power quality data to a designated category and partition;
determining an offset of the power quality data based on an offset at the end of the specified category and the partition.
4. The method for fusing power quality data according to any one of claims 1 to 3, wherein the method further comprises:
performing redundant backup on the power quality data and the corresponding offset in a kafka server cluster mode;
and determining the kafka server in the kafka server cluster to provide the service for the WEB client according to the service state of the partition in the kafka server.
5. The method for fusing power quality data according to claim 1, wherein the method further comprises:
and the kafka server deletes the electric energy quality data stored earlier according to a preset caching strategy and the storage sequence of the electric energy quality data.
6. A fusion device of electric energy quality data is characterized in that the fusion device of the electric energy quality data comprises:
the subscription data sending unit is used for acquiring power quality data by the power quality monitoring point and sending the power quality data to the kafka server in real time according to a subscription instruction;
the consumption request receiving unit is used for receiving a data consumption request of the data WEB client by the Kafka server;
the offset data sending unit is used for searching an offset corresponding to a data consumption request of the WEB client by the kafka server according to the data consumption record of the WEB client, acquiring the electric energy quality data corresponding to the offset from the kafka server, and sending the electric energy quality data to the WEB client;
the kafka server comprises a server cluster formed by a plurality of servers, the plurality of servers comprise a main server and a standby server, the same power quality data and offset corresponding to the power quality data are backed up on different servers, when the kafka server receives a data demand, a corresponding type and a corresponding partition are searched according to the offset, whether the partition is providing data transmission service or not is judged, and the data transmission service is provided for the WEB client through the type and the partition in an idle state.
7. The fusion device of the power quality data according to claim 6, wherein the subscription data sending unit includes:
the acquisition subunit is used for acquiring the power quality data in real time through the power quality monitoring points;
the instruction sending subunit is used for receiving a subscription instruction sent by the kafka server through a WEB service interface;
and the subscription message sending subunit is used for packaging the power quality data into a message and sending the message to the kafka server in real time, and the kafka server receives the message, decapsulates the message to obtain the power quality data and determines the offset of the received power quality data.
8. The fusion device of power quality data according to claim 7, wherein the subscription message sending subunit comprises:
the partition determining module is used for sending the power quality data to a specified category and a partition;
and the offset determining module is used for determining the offset of the power quality data on the basis of the offset of the tail of the specified category and the partition.
9. A fusion system of power quality data, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the fusion method of power quality data according to any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method for fusion of power quality data according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810368579.5A CN108805741B (en) | 2018-04-23 | 2018-04-23 | Fusion method, device and system of power quality data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810368579.5A CN108805741B (en) | 2018-04-23 | 2018-04-23 | Fusion method, device and system of power quality data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108805741A CN108805741A (en) | 2018-11-13 |
| CN108805741B true CN108805741B (en) | 2022-02-11 |
Family
ID=64093921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810368579.5A Active CN108805741B (en) | 2018-04-23 | 2018-04-23 | Fusion method, device and system of power quality data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108805741B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111723160B (en) * | 2020-08-24 | 2021-03-23 | 国网浙江省电力有限公司 | Multi-source heterogeneous incremental data synchronization method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201540592U (en) * | 2009-11-13 | 2010-08-04 | 上海市电力公司 | Electric energy quality monitoring management system structure |
| CN102435868A (en) * | 2010-09-29 | 2012-05-02 | 上海联能仪表有限公司 | Power quality monitoring system |
| CN102882274A (en) * | 2012-09-07 | 2013-01-16 | 中国电力科学研究院 | Terminal fusion system applied to distribution network |
| CN103366311A (en) * | 2013-07-11 | 2013-10-23 | 昆明能讯科技有限责任公司 | Data fusion processing method based on transformer substation multi-system |
| CN104092767A (en) * | 2014-07-21 | 2014-10-08 | 北京邮电大学 | Posting/subscribing system for adding message queue models and working method thereof |
| CN106651633A (en) * | 2016-10-09 | 2017-05-10 | 国网浙江省电力公司信息通信分公司 | Power utilization information acquisition system and method based on big data technology |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11005933B2 (en) * | 2016-03-17 | 2021-05-11 | International Business Machines Corporation | Providing queueing in a log streaming messaging system |
-
2018
- 2018-04-23 CN CN201810368579.5A patent/CN108805741B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201540592U (en) * | 2009-11-13 | 2010-08-04 | 上海市电力公司 | Electric energy quality monitoring management system structure |
| CN102435868A (en) * | 2010-09-29 | 2012-05-02 | 上海联能仪表有限公司 | Power quality monitoring system |
| CN102882274A (en) * | 2012-09-07 | 2013-01-16 | 中国电力科学研究院 | Terminal fusion system applied to distribution network |
| CN103366311A (en) * | 2013-07-11 | 2013-10-23 | 昆明能讯科技有限责任公司 | Data fusion processing method based on transformer substation multi-system |
| CN104092767A (en) * | 2014-07-21 | 2014-10-08 | 北京邮电大学 | Posting/subscribing system for adding message queue models and working method thereof |
| CN106651633A (en) * | 2016-10-09 | 2017-05-10 | 国网浙江省电力公司信息通信分公司 | Power utilization information acquisition system and method based on big data technology |
Non-Patent Citations (1)
| Title |
|---|
| 基于Hadoop的海量电能质量监测数据云平台的研究;周艳玲;《中国优秀硕士学位论文全文数据库-工程科技Ⅱ辑》;20180115;摘要，第8-46页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108805741A (en) | 2018-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111752799A (en) | Service link tracking method, device, equipment and storage medium | |
| CN111258978B (en) | Data storage method | |
| WO2021151312A1 (en) | Method for determining inter-service dependency, and related apparatus | |
| CN102957622B (en) | Method, device and system for data processing | |
| CN111212135A (en) | Message subscription method, device, system, electronic equipment and storage medium | |
| CN110928681A (en) | Data processing method and device, storage medium and electronic device | |
| CN105373563B (en) | Database switching method and device | |
| CN109788251B (en) | Video processing method, device and storage medium | |
| CN114676205A (en) | Event information processing method and device | |
| CN108805741B (en) | Fusion method, device and system of power quality data | |
| CN112417050A (en) | Data synchronization method and device, system, storage medium and electronic device | |
| CN111309693A (en) | Data synchronization method, device and system, electronic equipment and storage medium | |
| CN115086347B (en) | Data synchronization method, device, electronic equipment and storage medium | |
| CN111291127B (en) | Data synchronization method, device, server and storage medium | |
| CN111400327B (en) | Data synchronization method and device, electronic equipment and storage medium | |
| CN104731660A (en) | Data distribution method, device and system | |
| CN114490718A (en) | Data output method, data output device, electronic equipment and computer readable medium | |
| CN112055058A (en) | Data storage method and device and computer readable storage medium | |
| CN112597119A (en) | Method and device for generating processing log and storage medium | |
| CN113515425A (en) | Method and device for dynamically reporting data | |
| CN113032477A (en) | Long-distance data synchronization method and device based on GTID and computing equipment | |
| CN103856359A (en) | Method and system for obtaining information | |
| CN116610643B (en) | Method, device and program product for monitoring directory compression downloading process | |
| CN112311833B (en) | Data updating method and device | |
| CN103107991B (en) | A kind of many NVT event-handling method, NVC and safety monitoring platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |