CN114286203A - Electric meter nuclear data shunting processing system and method based on big data - Google Patents
Electric meter nuclear data shunting processing system and method based on big data Download PDFInfo
- Publication number
- CN114286203A CN114286203A CN202111581964.6A CN202111581964A CN114286203A CN 114286203 A CN114286203 A CN 114286203A CN 202111581964 A CN202111581964 A CN 202111581964A CN 114286203 A CN114286203 A CN 114286203A
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- response rate
- unit
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 99
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000004044 response Effects 0.000 claims abstract description 46
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 238000004458 analytical method Methods 0.000 claims abstract description 7
- 230000010354 integration Effects 0.000 claims abstract description 7
- 238000003672 processing method Methods 0.000 claims abstract 2
- 230000002950 deficient Effects 0.000 claims description 6
- 238000002203 pretreatment Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides an ammeter nuclear receiving data shunting processing method based on big data, which comprises the following steps: step S10, periodically collecting the operation data of various electric meter related devices; step S11, preprocessing the collected operation data and standardizing each operation data; step S12, performing data classification on all the preprocessed data, and respectively importing the data into a plurality of data processing channels; step S13, determining whether the data of each channel needs to be distributed according to the real-time response rate of the system; step S14, when needing to shunt, the data needed to shunt is shunted to the corresponding parallel server; otherwise, transmitting the data to the main server; in step S15, the main server or the parallel server performs an integration analysis and a verification process on the data to form a user data packet for each user. The invention also discloses a corresponding system. By implementing the invention, the meter reading efficiency can be improved, and the overall time for collecting and checking the ammeter data is shortened.
Description
Technical Field
The invention relates to the technical field of electric meter verification and collection, in particular to an electric meter verification and collection data shunting processing system and method based on big data.
Background
With the vigorous development of digital technologies and applications such as 'cloud big thing moving intelligence', power enterprises in recent years continuously perfect a power marketing system on the premise of ensuring power stability. The electricity charge reading, checking and receiving work is the key part work of an electric power enterprise, the meter reading is the data acquisition link of a reading, checking and receiving system, and the data acquisition link is the basis of the following two works. In the prior art, meter reading in a reading-checking-receiving system is realized by a task uploading and data complementary collection mode, but a hundred thousand-level terminal device is simultaneously connected with a main station front-end processor, so that pressure is brought to the connection efficiency of the front-end processor, and in the transmission process after data collection, data need to pass through a safety access area and reach a subsequent device through two isolation devices. The existing integral meter reading time is long, the data acquisition and data transmission have an optimized space, and the meter reading efficiency is low.
Disclosure of Invention
The technical problem to be solved by the invention is to provide the ammeter data collecting and receiving shunting processing system and method based on the big data, which can improve the meter reading efficiency and shorten the overall time for collecting and receiving ammeter data.
In order to solve the above technical problems, an aspect of the present invention provides a method for shunting data received by an electric meter core based on big data, which includes the following steps:
step S10, periodically collecting the operation data of the related equipment of various electric meters through the front-end processor;
step S11, preprocessing the collected operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data;
step S12, determining the data type of each preprocessed data, dividing all the preprocessed data according to the data type, and respectively importing the data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
step S13, determining whether the data of each data processing channel needs to be distributed according to the real-time response rate of the system;
step S14, if the judgment result is that the data needs to be distributed, distributing the data of the data processing channel needing to be distributed to one of a plurality of parallel servers connected with the main server; otherwise, transmitting the data of the data processing channel to a main server;
step S15, after receiving the data of the data processing channel, the main server or the parallel server carries out integration analysis and check processing on the data to form a user data packet of each user; and pushing the user data packet to a corresponding user side.
Preferably, further comprising: the method comprises the steps of configuring the corresponding relation between parallel servers and data categories in advance, wherein each parallel server corresponds to one data category.
Preferably, the step S13 further includes:
comparing the real-time response rate of the system with a preset response rate threshold, and if the real-time response rate of the system is smaller than the response threshold, determining that shunting processing is required; otherwise, determining that the shunting processing is not needed;
if the data needs to be distributed, inquiring a corresponding table of response rate and distribution quantity according to the real-time response rate, determining the quantity of the data processing channels needing to be distributed, and obtaining the parallel server information corresponding to each data processing channel so as to distribute the data of each data processing channel needing to be distributed to the corresponding parallel server.
Correspondingly, in another aspect of the present invention, a big data based electric meter nuclear receiving data shunting processing system is further provided, which includes:
the data acquisition unit is used for periodically acquiring the operation data of various types of electric meter related equipment through the front-end processor;
the preprocessing unit is used for preprocessing the acquired operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data;
the data dividing unit is used for determining the data type of each preprocessed data, dividing all the preprocessed data according to the data type and respectively importing the data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
the shunt judgment unit is used for determining whether the data of each data processing channel needs to be shunted or not according to the real-time response rate of the system;
the distribution processing unit is used for distributing the data of the data processing channel needing distribution to one of a plurality of parallel servers connected with the main server if the judgment result of the distribution judging unit is that the distribution is needed; otherwise, transmitting the data of the data processing channel to a main server;
the checking processing unit is used for performing integration analysis and checking processing on the data after the main server or the parallel server receives the data of the data processing channel to form a user data packet of each user; and pushing the user data packet to a corresponding user side.
Preferably, further comprising:
the configuration unit is used for configuring the corresponding relation between the parallel servers and the data categories in advance, wherein each parallel server corresponds to one data category.
Preferably, the shunt judgment unit further includes:
the comparison unit is used for comparing the real-time response rate of the system with a preset response rate threshold value, and if the real-time response rate of the system is smaller than the response threshold value, the shunt processing is determined to be required; otherwise, determining that the shunting processing is not needed;
and the flow dividing channel number determining unit is used for inquiring a corresponding table of response rate and flow dividing number according to the real-time response rate when the comparing unit determines that flow dividing processing is needed, determining the number of the data processing channels needing flow dividing, and acquiring the parallel server information corresponding to each data processing channel.
The embodiment of the invention has the following beneficial effects:
the invention provides a big data-based ammeter nuclear receiving data shunting processing system and method, which can enable different servers to process different types of ammeter data in parallel by shunting the collected massive ammeter data, thereby being capable of rapidly performing nuclear receiving processing on the ammeter data and greatly improving the overall data processing efficiency; the whole time for collecting and checking the ammeter data is greatly shortened.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is within the scope of the present invention for those skilled in the art to obtain other drawings based on the drawings without inventive exercise.
Fig. 1 is a schematic main flow chart of an embodiment of a split-flow processing method for electric meter acceptance data based on big data according to the present invention;
fig. 2 is a schematic structural diagram of a big data-based electric meter verification data distribution processing system provided by the present invention;
fig. 3 is a schematic structural diagram of the flow determination unit in fig. 2.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings.
Fig. 1 is a main flow diagram illustrating an embodiment of a method for shunting data received by an electric meter based on big data according to the present invention; in the present embodiment, it is preferred that,
in order to solve the above technical problems, an aspect of the present invention provides a method for shunting data received by an electric meter core based on big data, which includes the following steps:
step S10, periodically collecting the operation data of the related equipment of various electric meters through the front-end processor; it can be understood that, due to different types of the related devices of the electric meter, the data formats acquired by the electric meter are different;
step S11, preprocessing the collected operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data; and controlling the length of the acquired data to realize data standardization. To facilitate processing in subsequent steps;
step S12, determining the data type of each preprocessed data, dividing all the preprocessed data according to the data type, and respectively importing the data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
step S13, determining whether the data of each data processing channel needs to be distributed according to the real-time response rate of the system;
specifically, in a specific example, preferably, the step S13 further includes:
comparing the real-time response rate of the system with a preset response rate threshold, and if the real-time response rate of the system is smaller than the response threshold (such as 50%), determining that shunting processing is required; otherwise, determining that the shunting processing is not needed;
if the data needs to be distributed, inquiring a corresponding table of response rate and distribution quantity according to the real-time response rate, determining the quantity of the data processing channels needing to be distributed, and obtaining the parallel server information corresponding to each data processing channel so as to distribute the data of each data processing channel needing to be distributed to the corresponding parallel server. Specifically, the lower the real-time response rate is, which indicates that the real-time processing efficiency of the existing system is lower, the more data of the data processing channels need to be distributed to the parallel server for processing; the relationship (including specific channels) between the real-time response rate and the shunt number in the response rate and shunt number correspondence table can be calibrated or configured in advance.
It can be understood that, in the method of the present invention, the correspondence between the parallel servers and the data categories needs to be configured in advance, wherein each parallel server corresponds to one data category.
Step S14, if the judgment result is that the data needs to be distributed, distributing the data of the data processing channel needing to be distributed to one of a plurality of parallel servers connected with the main server; otherwise, transmitting the data of the data processing channel to a main server;
step S15, after receiving the data of the data processing channel, the main server or the parallel server carries out integration analysis and check processing on the data to form a user data packet of each user; and pushing the user data packet to a corresponding user side. Specifically, in this step, at least the resources in the database may be configured and integrated according to a preset distribution scheme to obtain a data packet, and the data packet is distributed and pushed to the user side.
Fig. 2 is a schematic structural diagram illustrating an embodiment of a big data-based electric meter core data distribution processing system according to the present invention. Referring to fig. 3 together, in this embodiment, the system 1 at least includes:
the data acquisition unit 10 is used for periodically acquiring the operation data of various electric meter related devices through the front-end processor;
the preprocessing unit 11 is used for preprocessing the acquired operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data;
the data dividing unit 12 is configured to determine a data type of each preprocessed data, divide all the preprocessed data according to the data type, and import the divided data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
the shunt judgment unit 13 is configured to determine whether data of each data processing channel needs to be shunted according to the system real-time response rate;
the distribution processing unit 14 is configured to distribute data of the data processing channel to be distributed to one of the multiple parallel servers connected to the main server if the determination result of the distribution determining unit is that distribution is required; otherwise, transmitting the data of the data processing channel to a main server;
a checking processing unit 15, configured to perform, after the main server or the parallel server receives data of the data processing channel, integration analysis and checking processing on the data to form a user data packet of each user; pushing the user data packet to a corresponding user side;
the configuration unit 16 is configured to pre-configure a corresponding relationship between parallel servers and data categories, where each parallel server corresponds to one data category.
As shown in fig. 3, the shunt determination unit 14 further includes:
the comparing unit 140 is configured to compare the system real-time response rate with a preset response rate threshold, and if the system real-time response rate is smaller than the response threshold, determine that the shunting processing is required; otherwise, determining that the shunting processing is not needed;
the shunting channel number determining unit 141 is configured to, when the comparing unit determines that shunting processing is required, query a response rate and shunting number correspondence table according to the real-time response rate, determine the number of data processing channels that need to be shunted, and obtain parallel server information corresponding to each data processing channel.
For more details, reference may be made to and combined with the foregoing description of fig. 1, which is not repeated herein.
The embodiment of the invention has the following beneficial effects:
the invention provides a big data-based ammeter nuclear receiving data shunting processing system and method, which can enable different servers to process different types of ammeter data in parallel by shunting the collected massive ammeter data, thereby being capable of rapidly performing nuclear receiving processing on the ammeter data and greatly improving the overall data processing efficiency; the whole time for collecting and checking the ammeter data is greatly shortened.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (6)
1. An electric meter nuclear data shunting processing method based on big data is characterized by comprising the following steps:
step S10, periodically collecting the operation data of the related equipment of various electric meters through the front-end processor;
step S11, preprocessing the collected operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data;
step S12, determining the data type of each preprocessed data, dividing all the preprocessed data according to the data type, and respectively importing the data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
step S13, determining whether the data of each data processing channel needs to be distributed according to the real-time response rate of the system;
step S14, if the judgment result is that the data needs to be distributed, distributing the data of the data processing channel needing to be distributed to one of a plurality of parallel servers connected with the main server; otherwise, transmitting the data of the data processing channel to a main server;
step S15, after receiving the data of the data processing channel, the main server or the parallel server carries out integration analysis and check processing on the data to form a user data packet of each user; and pushing the user data packet to a corresponding user side.
2. The method of claim 1, further comprising:
the method comprises the steps of configuring the corresponding relation between parallel servers and data categories in advance, wherein each parallel server corresponds to one data category.
3. The method of claim 2, wherein the step S13 further comprises:
comparing the real-time response rate of the system with a preset response rate threshold, and if the real-time response rate of the system is smaller than the response threshold, determining that shunting processing is required; otherwise, determining that the shunting processing is not needed;
if the data needs to be distributed, inquiring a corresponding table of response rate and distribution quantity according to the real-time response rate, determining the quantity of the data processing channels needing to be distributed, and obtaining the parallel server information corresponding to each data processing channel so as to distribute the data of each data processing channel needing to be distributed to the corresponding parallel server.
4. The utility model provides an ammeter nuclear receipt data reposition of redundant personnel processing system based on big data which characterized in that includes:
the data acquisition unit is used for periodically acquiring the operation data of various types of electric meter related equipment through the front-end processor;
the preprocessing unit is used for preprocessing the acquired operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data;
the data dividing unit is used for determining the data type of each preprocessed data, dividing all the preprocessed data according to the data type and respectively importing the data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
the shunt judgment unit is used for determining whether the data of each data processing channel needs to be shunted or not according to the real-time response rate of the system;
the distribution processing unit is used for distributing the data of the data processing channel needing distribution to one of a plurality of parallel servers connected with the main server if the judgment result of the distribution judging unit is that the distribution is needed; otherwise, transmitting the data of the data processing channel to a main server;
the checking processing unit is used for performing integration analysis and checking processing on the data after the main server or the parallel server receives the data of the data processing channel to form a user data packet of each user; and pushing the user data packet to a corresponding user side.
5. The system of claim 4, further comprising:
the configuration unit is used for configuring the corresponding relation between the parallel servers and the data categories in advance, wherein each parallel server corresponds to one data category.
6. The system of claim 2, wherein the diversion determination unit further comprises:
the comparison unit is used for comparing the real-time response rate of the system with a preset response rate threshold value, and if the real-time response rate of the system is smaller than the response threshold value, the shunt processing is determined to be required; otherwise, determining that the shunting processing is not needed;
and the flow dividing channel number determining unit is used for inquiring a corresponding table of response rate and flow dividing number according to the real-time response rate when the comparing unit determines that flow dividing processing is needed, determining the number of the data processing channels needing flow dividing, and acquiring the parallel server information corresponding to each data processing channel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111581964.6A CN114286203B (en) | 2021-12-22 | 2021-12-22 | Electric meter nuclear data distribution processing system and method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111581964.6A CN114286203B (en) | 2021-12-22 | 2021-12-22 | Electric meter nuclear data distribution processing system and method based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114286203A true CN114286203A (en) | 2022-04-05 |
| CN114286203B CN114286203B (en) | 2024-03-29 |
Family
ID=80873814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111581964.6A Active CN114286203B (en) | 2021-12-22 | 2021-12-22 | Electric meter nuclear data distribution processing system and method based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114286203B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673457A (en) * | 2009-08-14 | 2010-03-17 | 深圳市科陆电子科技股份有限公司 | Method and system of data acquisition from data acquisition terminal |
| CN102510568A (en) * | 2011-11-22 | 2012-06-20 | 联通宽带业务应用国家工程实验室有限公司 | Internet access data processing system and method for mobile terminal |
| CN103886735A (en) * | 2014-03-07 | 2014-06-25 | 北京国电通网络技术有限公司 | Acquisition method and acquisition system of power utilization information, and meter reading equipment |
| CN104112345A (en) * | 2014-07-29 | 2014-10-22 | 深圳华越南方电子技术有限公司 | Data monitoring method of remote wireless meter reading system |
| CN108563664A (en) * | 2018-01-04 | 2018-09-21 | 上海电气集团股份有限公司 | A kind of real-time data processing method at industrial equipment end |
| CN112650788A (en) * | 2020-12-25 | 2021-04-13 | 青岛鼎信通讯股份有限公司 | Power consumption behavior monitoring system based on non-invasive ammeter |
-
2021
- 2021-12-22 CN CN202111581964.6A patent/CN114286203B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673457A (en) * | 2009-08-14 | 2010-03-17 | 深圳市科陆电子科技股份有限公司 | Method and system of data acquisition from data acquisition terminal |
| CN102510568A (en) * | 2011-11-22 | 2012-06-20 | 联通宽带业务应用国家工程实验室有限公司 | Internet access data processing system and method for mobile terminal |
| CN103886735A (en) * | 2014-03-07 | 2014-06-25 | 北京国电通网络技术有限公司 | Acquisition method and acquisition system of power utilization information, and meter reading equipment |
| CN104112345A (en) * | 2014-07-29 | 2014-10-22 | 深圳华越南方电子技术有限公司 | Data monitoring method of remote wireless meter reading system |
| CN108563664A (en) * | 2018-01-04 | 2018-09-21 | 上海电气集团股份有限公司 | A kind of real-time data processing method at industrial equipment end |
| CN112650788A (en) * | 2020-12-25 | 2021-04-13 | 青岛鼎信通讯股份有限公司 | Power consumption behavior monitoring system based on non-invasive ammeter |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114286203B (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110674189B (en) | Method for monitoring secondary state and positioning fault of intelligent substation | |
| CN108647328B (en) | Reliability power failure event acquisition integrity analysis and automatic completion method | |
| CN106530121B (en) | Method and system for detecting safety protection compliance of power monitoring system | |
| CN109494757B (en) | Voltage reactive power operation early warning method and system | |
| CN114880312B (en) | Flexibly-set application system service data auditing method | |
| CN112615428A (en) | Line loss analysis and treatment system and method | |
| CN115272027A (en) | Low-carbon emission data monitoring system for double-carbon management | |
| CN112365186A (en) | Health degree evaluation method and system for electric power information system | |
| CN116934270A (en) | Library book borrowing management system based on data analysis | |
| CN106022640B (en) | Electric quantity index checking system and method | |
| CN108093039B (en) | Method for processing sleeping electric meter based on national network Q/GDW3761 protocol | |
| CN106997487B (en) | Knowledge base-based aircraft test data automatic interpretation method | |
| CN103632522A (en) | Mobile phone meter reading system | |
| CN108613820A (en) | A kind of online allophone monitoring algorithm for GIS bulk mechanicals defect diagonsis and positioning | |
| CN114286203B (en) | Electric meter nuclear data distribution processing system and method based on big data | |
| CN108989116B (en) | Big data collection and analysis system and method | |
| CN108376324B (en) | System and method for managing metering assets of carrier collector | |
| CN115801828A (en) | Intelligent substation secondary data analysis system based on platform and terminal | |
| CN104835313A (en) | Electricity consumption information acquisition device data transmission performance analysis method | |
| CN114064622A (en) | Big data-based power failure study and judgment processing method and system | |
| CN110348984B (en) | Automatic credit card data input method and related equipment under different transaction channels | |
| CN113691390A (en) | Cloud-end-coordinated edge node alarm system and method | |
| CN112560908A (en) | Cloud-end cooperative load identification system and method | |
| CN110515955A (en) | Storage, querying method, system, electronic equipment and the storage medium of data | |
| CN113810316B (en) | Intelligent substation switch mirror image configuration method and device based on SCD file |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |