CN111694819A - Electric power abnormal data filtering method and device based on sub-bit distance algorithm - Google Patents
Electric power abnormal data filtering method and device based on sub-bit distance algorithm Download PDFInfo
- Publication number
- CN111694819A CN111694819A CN202010342686.8A CN202010342686A CN111694819A CN 111694819 A CN111694819 A CN 111694819A CN 202010342686 A CN202010342686 A CN 202010342686A CN 111694819 A CN111694819 A CN 111694819A
- Authority
- CN
- China
- Prior art keywords
- power data
- quantile
- data
- analyzed
- quantiles
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 38
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 23
- 238000002372 labelling Methods 0.000 claims abstract description 11
- 230000011218 segmentation Effects 0.000 claims description 21
- 230000005856 abnormality Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Abstract
The embodiment of the application relates to a power abnormal data filtering method and device based on a sub-bit distance algorithm, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring power data to be analyzed; reading the power data which is filtered last time from the database; calculating quantiles and quantile distances corresponding to the power data to be analyzed according to the power data which is filtered last time; and filtering the electric power data to be analyzed according to the quantiles and the quantile distances, and labeling the electric power data with abnormity. The electric power abnormal data filtering method and device based on the sub-bit distance algorithm, the electronic equipment and the computer readable storage medium can accurately filter abnormal electric power data and improve the data processing efficiency.
Description
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a method and a device for filtering abnormal power data based on a sub-bit distance algorithm, electronic equipment and a computer readable storage medium.
Background
In order to better allocate power resources, data collection and analysis of power resources used by each user are required to make a better power resource allocation plan. At present, due to the influence of external factors such as electromagnetic interference and noise on the acquisition device, some abnormal data inevitably exists in the acquired power data, and the abnormal power data adversely affects the subsequent analysis, so that it is necessary to perform abnormal value filtering on the acquired data. The traditional abnormal data filtering methods such as a distance identification method and a density identification method generally have the defects of large calculated amount, complex algorithm and the like, and are not beneficial to large-scale data processing.
Disclosure of Invention
The embodiment of the application provides a power abnormal data filtering method and device based on a sub-bit distance algorithm, an electronic device and a computer readable storage medium, which can accurately filter abnormal power data and improve data processing efficiency.
A power abnormal data filtering method based on a sub-bit distance algorithm comprises the following steps:
acquiring power data to be analyzed;
reading the power data which is filtered last time from the database;
calculating quantiles and quantile distances corresponding to the power data to be analyzed according to the power data which is filtered last time;
and filtering the electric power data to be analyzed according to the quantiles and the quantile distances, and labeling the electric power data with abnormity.
An electric power abnormal data filtering device based on a sub-bit distance algorithm comprises:
the acquisition module is used for acquiring power data to be analyzed;
the reading module is used for reading the power data which is filtered last time from the database;
the calculation module is used for calculating the quantiles and the quantile distances corresponding to the power data to be analyzed according to the power data which is filtered last time;
and the filtering module is used for filtering the electric power data to be analyzed according to the quantiles and the quantile distances and marking the electric power data with abnormity.
An electronic device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to carry out the method as described above.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method as set forth above.
The power abnormal data filtering method and device based on the quantile distance algorithm, the electronic equipment and the computer readable storage medium acquire power data to be analyzed, read power data filtered last time from a database, calculate the quantile and the quantile distance corresponding to the power data to be analyzed according to the power data filtered last time, filter the power data to be analyzed according to the quantile and the quantile distance, mark the power data with abnormality, filter the power data by using the quantile and the quantile distance, and accurately eliminate the power data with abnormality. And the quantiles and the quantile distances to be analyzed can be calculated according to the power data filtered last time, so that the accuracy of data filtering is improved, and the data filtering efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a diagram illustrating an application scenario of a power anomaly data filtering method based on a fractional bit distance algorithm according to an embodiment;
FIG. 2 is a flow diagram of a power anomaly data filtering method based on a fractional bit distance algorithm in one embodiment;
FIG. 3 is a flow chart of a power anomaly data filtering method based on a bit-spacing algorithm according to another embodiment;
FIG. 4 is a flow diagram of filtering power data to be analyzed according to quantiles and quantiles in one embodiment;
FIG. 5 is a block diagram of a power anomaly data filtering device based on a fractional bit distance algorithm in one embodiment;
FIG. 6 is a block diagram of an electronic device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first client may be referred to as a second client, and similarly, a second client may be referred to as a first client, without departing from the scope of the present application. Both the first client and the second client are clients, but they are not the same client.
Fig. 1 is an application scenario diagram of a power anomaly data filtering method based on a fractional bit distance algorithm in an embodiment. As shown in fig. 1, the server 20 may establish a communication connection with one or more collection devices 10, and the collection devices 10 may collect power data of the electric meters and upload the collected power data to the server 20 for storage. The server 20 may store and process the collected power data, such as, but not limited to, data filtering, data cleansing, data conversion, and the like. In some embodiments, the server 20 may also perform analysis on the processed power data, such as market trend analysis using the power data, comprehensive evaluation of customers, and so forth. The server 20 may provide technical support, application services, and the like to power grid companies, power generation companies, power selling companies, and the like based on the analysis of the power data, wherein the application services may include, but are not limited to, load prediction, power price prediction, energy saving optimization, and the like.
In the embodiment of the present application, the server 20 may filter the collected power data to filter out abnormal data in the power data. The server 20 may obtain the power data to be analyzed, read the power data that has been filtered last time from the database, and calculate the quantile and the quantile distance corresponding to the power data to be analyzed according to the power data that has been filtered last time. The server 20 may filter the power data to be analyzed according to the quantile and the quantile distance, and mark the power data with an abnormal condition.
As shown in fig. 2, in an embodiment, a power anomaly data filtering method based on a sub-bit distance algorithm is provided, which can be applied to the server, and the method can include the following steps:
The collection device collects the power data of the ammeter and sends the power data to the server for storage. The server may obtain power data to be analyzed, wherein the power data may include, but is not limited to, voltage, current, power, and the like. In one embodiment, the collecting device may upload power data to the server at preset time intervals, for example, every 4 hours, 10 hours, 1 day, and the like, where the power data is power data generated by the electric meter within a time duration corresponding to the preset time interval. The server receives the electric power data uploaded by the acquisition device, can take the uploaded electric power data as the electric power data to be analyzed, carries out data filtering, and stores the filtered data into the database, so that the stored data volume can be reduced, and the accuracy of subsequent data analysis can be improved.
The server can read the power data which is filtered last time from the database, and calculate the quantile and the quantile distance corresponding to the power data to be analyzed this time according to the read power data which is filtered last time. In one embodiment, each time the server filters the power data, the filtered power data may be stored in the database and the corresponding storage time may be recorded. The server can read the preset amount of power data stored in the database at the latest time. Alternatively, the preset number may be set according to actual requirements, for example, 400, 300, 470, and the like.
And step 230, calculating the quantiles and the quantile distances corresponding to the electric power data to be analyzed according to the last filtered electric power data.
The server can filter the power data to be analyzed by using a quantile filtering method. The server can determine the quantile and the quantile distance corresponding to the electric power data to be analyzed. The quantile is a numerical point that divides the probability distribution range of a random variable into several equal parts, and the quantile may be a median (i.e., a binary), a quartile, a percentile, or the like, and is not limited herein. The quantile distance refers to a difference between two quantiles, for example, a difference between a first quantile and a third quantile in a quartile, a difference between a sixth quantile and a third quantile in a decile, and the like, but is not limited thereto. Discrete trends in the data may be measured by fractional bit distance.
In some embodiments, the quantile and the quantile distance corresponding to the power data to be analyzed may be fixed values stored in advance, or may be calculated according to the power data collected in real time, or may be calculated according to the latest power data stored in a database, where the latest power data stored in the database may be the power data that has been filtered by the server last time. The manner of obtaining the quantiles and the quantiles is not limited herein.
And 240, filtering the electric power data to be analyzed according to the quantiles and the quantiles, and labeling the electric power data with abnormality.
In some embodiments, the server may calculate a critical value of the power data to be analyzed according to the quantile and the quantile distance, and filter the power data to be analyzed by using the critical value. If the power data to be analyzed is within the critical value, the power data may be determined to be normal data. If the power data to be analyzed is outside the critical value, the power data outside the critical value may be labeled as abnormal data. The electric power data marked as abnormal data can be removed, and data filtering can be completed after the removal is completed. The server can store the power data after data filtering in the database for subsequent data processing and analysis, such as data conversion, market trend analysis by using the power data, and the like, so that the amount of stored data can be reduced, and the accuracy of data processing and analysis can be improved.
In the embodiment of the application, the electric power data to be analyzed is obtained, the electric power data which is filtered last time is read from the database, the quantiles and the quantile distances corresponding to the electric power data to be analyzed are calculated according to the electric power data which is filtered last time, the electric power data to be analyzed are filtered according to the quantiles and the quantile distances, the electric power data with the abnormity are marked, the electric power data are filtered by the quantiles and the quantile distances, and the electric power data with the abnormity are accurately rejected. And the quantiles and the quantile distances to be analyzed can be calculated according to the power data filtered last time, so that the accuracy of data filtering is improved, and the data filtering efficiency is improved.
As shown in fig. 3, in an embodiment, the power anomaly data filtering method based on the fractional bit distance algorithm further includes the following steps:
In one embodiment, the power data to be analyzed may include at least one category of power data, which may include, but is not limited to, current class, voltage class, power class, electrical class, and the like. Different categories may correspond to different quantiles and quantiles. The types included in the electric power data to be analyzed can be determined, quantiles and quantiles corresponding to the electric power data of the types are obtained, and then the electric power data of the corresponding types are filtered according to the quantiles and the quantiles.
In some embodiments, the server may also filter a preset category of power data to be analyzed, where the preset category may be one or more of a plurality of categories. In one embodiment, the predetermined category may include at least one of a current category and a voltage category. The server can respectively determine the quantiles and the quantiles distances corresponding to the preset categories, further, the server can determine the quantiles and the quantiles distances corresponding to the voltage type electric power data, and can also determine the quantiles and the quantiles distances corresponding to the current type electric power data. It is understood that the preset category may also be other categories, and is not limited to the above categories, and may be set according to actual requirements.
And step 304, reading the power data which is filtered last time and is matched with each determined category from the database.
After the server determines the categories contained in the power data to be analyzed, the server can read the power data which is filtered last time and corresponds to the contained categories from the database, and calculate the quantiles and the quantile distances corresponding to the categories according to the read power data.
In one embodiment, each time the server filters the power data, the filtered power data may be stored in the database and the corresponding storage time may be recorded. The server may read a fixed number of pieces of power data corresponding to each category included in the power data to be analyzed, which is stored in the database at the latest time. Alternatively, the fixed number may be set according to actual requirements, for example, 100 reads, 200 reads, etc. for each category.
And step 306, calculating quantiles and quantiles corresponding to the categories according to the power data matched with the categories.
The server can calculate the quantiles and the quantile distances corresponding to the categories according to the fixed amount of power data corresponding to the categories. In one embodiment, the fixed amount of power data may be arranged in order from small to large, and the power data corresponding to each sub-site may be determined.
In some embodiments, the server may divide the arranged power data into four equal parts, may determine three division points of the power data, and calculate power data corresponding to the three division points, that is, quantiles.
A fixed number of pieces of power data are arranged in descending order, and a data sequence X {1}, X {2}, X {3} … … X { N }, where N = a fixed number, is obtained. Three segmentation points may be determined first, where the first segmentation point a = (N + 1)/4, the second segmentation point B = (N + 1)/2, and the third segmentation point C =3 = (N + 1)/4.
The quantile corresponding to the second segmentation point can be determined according to the odd-even number condition of N. When N is an odd number, the quantile of the second division point is the power data corresponding to the second division point B, that is, the quantile M = X { B } of the second division point. When N is an even number, the integer part y1 of the second division point B is determined, and the quantile of the second division point B is calculated according to the integer part y1 of the second division point B, specifically, the quantile M = (X { y1} + X { y1+1})/2 of the second division point.
The server calculates the quantiles corresponding to the first division point, may first determine the integer part y2 and the fractional part z of the first division point a, and calculate the quantile corresponding to the first division point according to the integer part y2 and the fractional part z of the first division point a. Specifically, the quantile Q1 corresponding to the first segmentation point = X { y2} + z (X { y2+1} -X { y2 }).
The server calculates the quantiles corresponding to the third segmentation point, may first determine the integer part y3 and the fractional part p of the third segmentation point C, and calculate the quantile corresponding to the third segmentation point according to the integer part y3 and the fractional part p of the third segmentation point C. Specifically, the quantile Q3 corresponding to the third segmentation point = X { y3} + p (X { y3+1} -X { y3 }).
After the server calculates the quantiles, the quantile distances may be calculated according to the quantiles, and optionally, the quantile distances of the quartiles may be a difference between a third quartile Q3 and a first quartile Q1, specifically, the quantile distances QR = Q3-Q1. It is to be understood that the quantiles and quantiles are not limited to the quartiles, but may be median, percentile, and the like.
In some embodiments, the server filters the preset type of power data to be analyzed, may read the last filtered preset type of power data from the database, and then calculates the quantiles and the quantile distances of the corresponding types according to the different preset types of power data, respectively. Specifically, the server may read the current data of the last filtered fixed amount from the database, and calculate the quantiles and the quantile distances corresponding to the current-type power data according to the current data of the fixed amount. And reading the voltage data of the last filtered fixed quantity from the database, and calculating the quantiles and the quantile distances corresponding to the voltage type electric power data according to the voltage data of the fixed quantity.
And 308, filtering the electric power data to be analyzed of each category according to the quantiles and the quantiles corresponding to each category, and labeling the electric power data with abnormality.
The server can filter the power data to be analyzed of each category according to the quantiles and the quantile distances corresponding to each category. In one embodiment, as shown in FIG. 4, step 308 may include the steps of:
The server can calculate a critical value according to the quantile and the quantile distance, and detect whether abnormal data exist in the power data to be analyzed according to the critical value. In one embodiment, the threshold values may include a lower threshold value a and a lower threshold value b, and the normal range of the power data may be [ a, b ]. Specifically, the server may calculate the critical value according to the first quartile Q1, the third quartile Q3 and the quantile distance QR, wherein the lower critical value a = max (0, Q1-m × QR), the upper critical value b = Q3+ n × QR, m and n may be constants, and different classes of m and n may set different constant values.
As an embodiment, the server may calculate a lower critical value Ia and an upper critical value Ib corresponding to the current-based power data according to the first quartile IQ1, the third quartile IQ3 and the quantile IQR of the current-based power data, where the lower critical value Ia = max (0, IQ1-20 × IQR) and the upper critical value Ib = IQ3+10 × IQR.
As an embodiment, the server may calculate a lower threshold Ua and an upper threshold Ub corresponding to the current power data according to the first quartile UQ1, the third quartile UQ3, and the quantile UQR of the voltage power data, where the lower threshold Ua = max (0, UQ1-5 × UQR) and the upper threshold Ub = UQ3+8 × UQR. It is understood that m and n can be adjusted according to actual requirements, and are not limited to the above.
In step 404, it is determined whether the power data to be analyzed of each category exceeds the corresponding threshold, if yes, step 406 is executed, and if not, step 408 is executed.
The server can judge whether the electric power data to be analyzed exceed a critical value, namely, can judge whether the electric power data after preliminary filtering exceed a normal range [ a, b ], and when the electric power data exceed the critical value, the electric power data exceeding the critical value can be marked as abnormal electric power data and eliminated.
In some embodiments, the server may determine whether each category of power data exceeds a corresponding threshold. Specifically, the server may determine whether a current value in the power data to be analyzed exceeds a lower critical value Ia and an upper critical value Ib corresponding to the current-type power data. If the current value is lower than the lower critical value Ia or higher than the upper critical value Ib, the current value can be judged to be abnormal data, and the current value can be eliminated. The server can also judge whether the voltage value in the power data to be analyzed exceeds a lower critical value Ua and an upper critical value Ub corresponding to the voltage type power data. If the voltage value is lower than the lower critical value Ua or higher than the upper critical value Ub, the voltage value can be judged to be abnormal data, and the data can be eliminated.
In step 406, the power data exceeding the critical value is marked as abnormal power data.
The server can mark the power data exceeding the critical value as abnormal power data, eliminate the marked power data, complete the re-filtering of the power data, and store the re-filtered power data in the database. The server can calculate the quantile and the quantile distance corresponding to the power data to be analyzed collected next time according to the power data stored in the database after the filtering so as to filter the power data to be analyzed next time.
In step 408, the power data is determined to be normal data.
In the embodiment of the application, the electric power data to be analyzed can be filtered according to the quantiles and the quantile distances corresponding to all the categories, and the electric power data with abnormity can be accurately removed. And the quantiles and the quantile distances to be analyzed can be calculated according to the power data filtered last time, so that the accuracy of data filtering is improved, and the data filtering efficiency is improved.
In some embodiments, before filtering the power data to be analyzed according to the quantile and the quantile distance, the server may perform preliminary filtering on the power data to be analyzed according to a threshold condition. The power data may be preset with a threshold value of the normal range, and the threshold condition may refer to a condition exceeding the threshold value of the normal range. For example, if the threshold value of the normal range corresponding to the current is 0 to 10A (ampere), the threshold condition may be that the current value exceeds 10A, the threshold value of the normal range corresponding to the voltage is 0 to 277V (volt), and the threshold condition may be that the voltage exceeds 277V.
The server can judge whether the power data to be analyzed meet a threshold condition, when the threshold condition is met, the power data meeting the threshold condition is not in a threshold value of a normal range, the power data meeting the threshold condition can be determined to be abnormal data, and the abnormal data are removed. For example, the power data includes a current value 20A, the server may determine that the current value exceeds 10A, that is, a threshold condition is satisfied, and may eliminate the current value 20A.
In one embodiment, the power data to be analyzed may include at least one category of power data, which may include, but is not limited to, current class, voltage class, power class, electrical class, and the like. The different categories can correspond to different threshold conditions, the threshold conditions can be set according to actual requirements, the threshold conditions can also be obtained by adjusting in real time according to the power data collected by the server, and the server can adjust the threshold values in the threshold conditions according to the collected power data. The categories included in the power data to be analyzed can be determined, threshold conditions corresponding to the power data of each category are obtained, and then the power data of the corresponding category are preliminarily filtered according to the threshold conditions.
For example, the power data to be analyzed includes power data of voltage class and power class. The voltage values contained in the electric power data to be analyzed can be preliminarily filtered according to the threshold condition corresponding to the voltage class, if the voltage exceeds 277V, the voltage value exceeding 277V in the electric power data to be analyzed can be eliminated. The power values contained in the electric power data to be analyzed can be preliminarily filtered according to the threshold condition corresponding to the power class, and if the threshold condition corresponding to the power class is that the total power factor is greater than or equal to 0, the power values with the total power factor greater than or equal to 0 in the electric power data to be analyzed can be eliminated. The server performs preliminary filtering on the electric power data according to the threshold condition, and can filter numerical values obviously with abnormality in the electric power data. And (4) performing secondary filtering on the preliminarily filtered power data by using a quantile filtering method.
In the embodiment of the application, the abnormal power data are effectively filtered through the combination of the threshold condition and the quantile, and the data processing efficiency is improved.
In an embodiment, after the server obtains the power data to be analyzed, the server may determine the connection mode of the electric meter corresponding to the power data to be analyzed, and the connection mode of the electric meter may include, but is not limited to, a three-phase four-wire connection mode, a three-phase three-wire connection mode, and the like. Compared with a three-phase four-wire system wiring mode, the three-phase three-wire system wiring mode is free of a zero line, and the wiring mode of the ammeter is determined by the incoming wire and the electricity utilization property of a user.
When the ammeter wiring mode that electric power data to be analyzed correspond is three-phase four-wire system wiring mode, the server can carry out harmonic filtering to electric power data after the preliminary filtering. The waveform of the normal power data is a standard sine wave, for example, the waveform of the normal voltage and the waveform of the normal current are sine waves. If the waveform distortion occurs, the harmonic wave can be determined to be contained, and whether the harmonic wave is within the preset harmonic wave standard limit value can be judged. And if the harmonic wave exceeds the preset harmonic wave standard limit value, determining that the electric power data corresponding to the harmonic wave exceeding the preset harmonic wave standard limit value is abnormal data.
In some embodiments, the server may generate waveform data from the power data and decompose the waveform data. Alternatively, the waveform data may be decomposed by a fourier method or the like, and the harmonic content included in the decomposed waveform data may be determined. Wherein the harmonic content can be expressed in percentage. The preset harmonic standard limit value can be used for limiting whether the harmonic content is in a normal range, for example, if the preset harmonic standard limit value is 5%, the normal range is 0-5%. The server can judge whether the harmonic content is in a normal range according to a preset harmonic standard limit value, and if the harmonic content is not in the normal range, the power data corresponding to the harmonic content which is not in the normal range can be determined as abnormal data.
As a specific example, different types of power data may correspond to different harmonic standard limits, for example, current-type power data and voltage-type power data may correspond to different harmonic standard limits. Further, under the same category of power data, a plurality of different harmonic standard limits may be set according to different situations, for example, for different grid nominal voltages, a plurality of different harmonic standard limits may be set, for example, the grid nominal voltage is 110V, and the harmonic standard limit may be 2%. It is to be understood that the above-mentioned harmonic standard limit values are only used for illustrating the embodiments of the present application, and are not used for limiting the embodiments of the present application.
After the harmonic filtering is carried out on the three-phase four-wire system power data by the server, the power data after the harmonic filtering can be filtered again according to the quantile and the quantile distance, and the abnormal power data are marked. In some embodiments, when the electric meter wiring mode corresponding to the electric power data to be analyzed is a three-phase three-wire system wiring mode, the preliminarily filtered electric power data may not be subjected to harmonic filtering, and the preliminarily filtered electric power data is directly subjected to secondary filtering according to the quantile and the quantile distance.
In the embodiment of the application, different data filtering methods can be adopted for the electric power data according to the electric meter wiring mode corresponding to the electric power data to be analyzed, and the data filtering efficiency can be improved. When the ammeter wiring mode that the electric power data that wait to analyze correspond is three-phase four-wire system wiring mode, can carry out the harmonic to the electric power data after the prefiltering and filter, further improve the accuracy of data filtration.
In one embodiment, a power anomaly data filtering method based on a sub-bit distance algorithm is provided, and comprises the following steps:
and (1) acquiring power data to be analyzed.
And (2) reading the power data which is filtered last time from the database.
In one embodiment, step (2) comprises: determining a category corresponding to the power data to be analyzed; and reading the power data which is filtered last time and is matched with each determined category from the database.
And (3) calculating the quantiles and the quantile distances corresponding to the electric power data to be analyzed according to the last filtered electric power data.
In one embodiment, step (3) comprises: arranging the power data which are filtered last time according to a sequence from small to large to obtain a data sequence; segmenting the data sequence and determining the quantile corresponding to each segmentation point; and selecting two quantiles, and calculating the difference between the two quantiles to obtain the quantile distance.
In one embodiment, determining the quantile corresponding to each segmentation point comprises: determining each segmentation point according to the data quantity contained in the data sequence, and calculating the integer part and the fraction part of each segmentation point; and calculating quantiles corresponding to the segmentation points based on the integer part and the fraction part.
And (4) filtering the electric power data to be analyzed according to the quantiles and the quantiles, and labeling the electric power data with abnormality.
In one embodiment, step (3) comprises: calculating quantiles and quantiles corresponding to the categories according to the power data matched with the categories; the step (4) comprises the following steps: and filtering the electric power data to be analyzed of each category according to the quantiles and the quantiles corresponding to each category, and labeling the electric power data with abnormality.
In one embodiment, the filtering the power data to be analyzed of each category according to the quantile and the quantile distance corresponding to each category, and labeling the power data with abnormality includes: calculating a critical value corresponding to each category according to the quantile and the quantile distance corresponding to each category; judging whether the power data to be analyzed of each category exceeds a corresponding critical value; and marking the power data exceeding the critical value as abnormal power data.
In an embodiment, the power anomaly data filtering method based on the fractional bit distance algorithm further includes: storing the filtered power data in a database, and recording the storage time; the step (2) comprises the following steps: and reading the preset amount of power data with the latest storage time from the database.
In the embodiment of the application, the electric power data to be analyzed is obtained, the electric power data which is filtered last time is read from the database, the quantiles and the quantile distances corresponding to the electric power data to be analyzed are calculated according to the electric power data which is filtered last time, the electric power data to be analyzed are filtered according to the quantiles and the quantile distances, the electric power data with the abnormity are marked, the electric power data are filtered by the quantiles and the quantile distances, and the electric power data with the abnormity are accurately rejected. And the quantiles and the quantile distances to be analyzed can be calculated according to the power data filtered last time, so that the accuracy of data filtering is improved, and the data filtering efficiency is improved.
It should be understood that, although the steps in the respective flow charts described above are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the various flow diagrams described above may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternatingly with other steps or at least a portion of the sub-steps or stages of other steps.
As shown in fig. 5, in one embodiment, a power anomaly data filtering apparatus 500 based on a fractional bit distance algorithm is provided, which includes an obtaining module 510, a reading module 520, a calculating module 530 and a filtering module 540.
An obtaining module 510, configured to obtain power data to be analyzed.
And a reading module 520, configured to read the power data that has been filtered last time from the database.
The calculating module 530 is configured to calculate a quantile and a quantile distance corresponding to the power data to be analyzed according to the power data that has been filtered last time.
And the filtering module 540 is configured to filter the power data to be analyzed according to the quantile and the quantile distance, and label the power data with an abnormal condition.
In one embodiment, the power anomaly data filtering apparatus 500 based on the fractional bit distance algorithm further includes a storage module, configured to store the filtered power data in a database, and record a storage time.
The reading module 520 is further configured to read a preset amount of power data with the latest storage time from the database.
In the embodiment of the application, the electric power data to be analyzed is obtained, the electric power data which is filtered last time is read from the database, the quantiles and the quantile distances corresponding to the electric power data to be analyzed are calculated according to the electric power data which is filtered last time, the electric power data to be analyzed are filtered according to the quantiles and the quantile distances, the electric power data with the abnormity are marked, the electric power data are filtered by the quantiles and the quantile distances, and the electric power data with the abnormity are accurately rejected. And the quantiles and the quantile distances to be analyzed can be calculated according to the power data filtered last time, so that the accuracy of data filtering is improved, and the data filtering efficiency is improved.
In one embodiment, the reading module 520 includes a category determination unit and a reading unit.
And the category determining unit is used for determining the category corresponding to the power data to be analyzed.
And the reading unit is used for reading the power data which is filtered last time and is matched with each determined category from the database.
In one embodiment, the calculating module 530 is further configured to calculate a quantile and a quantile distance corresponding to each category according to the power data matched with each category.
The filtering module 540 is further configured to filter the power data to be analyzed of each category according to the quantile and the quantile distance corresponding to each category, and mark the power data with an abnormal condition.
In one embodiment, the filtering module 540 includes a calculating unit, a determining unit and a labeling unit.
And the calculating unit is used for calculating the critical value corresponding to each category according to the quantile and the quantile distance corresponding to each category.
And the judging unit is used for judging whether the electric power data to be analyzed of each category exceeds the corresponding critical value.
And the marking unit is used for marking the power data exceeding the critical value as abnormal power data.
In one embodiment, the calculation module 530 includes an arrangement unit, a division unit, and a sub-bit distance obtaining unit.
And the arrangement unit is used for arranging the power data which are filtered last time in a sequence from small to large to obtain a data sequence.
And the dividing unit is used for dividing the data sequence and determining the quantile corresponding to each dividing point.
In an embodiment, the dividing unit is further configured to determine each dividing point according to the number of data included in the data sequence, calculate an integer part and a fraction part of each dividing point, and calculate a quantile corresponding to the dividing point based on the integer part and the fraction part.
And the quantile distance acquisition unit is used for selecting two quantiles and calculating the difference between the two quantiles to obtain the quantile distance.
In the embodiment of the application, the electric power data to be analyzed can be filtered according to the quantiles and the quantile distances corresponding to all the categories, and the electric power data with abnormity can be accurately removed. And the quantiles and the quantile distances to be analyzed can be calculated according to the power data filtered last time, so that the accuracy of data filtering is improved, and the data filtering efficiency is improved.
FIG. 6 is a block diagram of an electronic device in one embodiment. As shown in fig. 6, in one embodiment, the electronic device 600 may be a server. The electronic device 600 may include one or more of the following components: a processor 610 and a memory 620, wherein one or more application programs may be stored in the memory 620 and configured to be executed by the one or more processors 610, the one or more programs configured to perform the methods as described above.
The processor 610 may include one or more processing cores. The processor 610 interfaces with various components throughout the electronic device 600 using various interfaces and circuitry to perform various functions of the electronic device 600 and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 620 and invoking data stored in the memory 620. Alternatively, the processor 610 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 610 may integrate one or a combination of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 610, but may be implemented by a communication chip.
The Memory 620 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 620 may be used to store instructions, programs, code sets, or instruction sets. The memory 620 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like. The storage data area may also store data created during use by the electronic device 600, and the like.
It is understood that the electronic device 600 may include more or less structural elements than those shown in the above structural block diagrams, and is not limited thereto.
In an embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when being executed by a processor, carries out the method as described in the above embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.
Any reference to memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A power abnormal data filtering method based on a sub-bit distance algorithm is characterized by comprising the following steps:
acquiring power data to be analyzed;
reading the power data which is filtered last time from the database;
calculating quantiles and quantile distances corresponding to the power data to be analyzed according to the power data which is filtered last time;
and filtering the electric power data to be analyzed according to the quantiles and the quantile distances, and labeling the electric power data with abnormity.
2. The method of claim 1, wherein the power data to be analyzed comprises at least one category of power data;
the reading of the last filtered power data from the database includes:
determining a category corresponding to the power data to be analyzed;
and reading the power data which is filtered last time and is matched with each determined category from the database.
3. The method according to claim 2, wherein the calculating the quantile and the quantile distance corresponding to the power data to be analyzed according to the power data filtered last time comprises:
calculating quantiles and quantile distances corresponding to the various categories according to the power data matched with the various categories;
the filtering the electric power data to be analyzed according to the quantiles and the quantile distances and marking the electric power data with abnormity comprises the following steps:
and filtering the electric power data to be analyzed of each category according to the quantiles and the quantile distances corresponding to each category, and labeling the electric power data with abnormity.
4. The method according to claim 3, wherein the filtering the power data to be analyzed of each category according to the quantile and the quantile distance corresponding to each category and labeling the power data with abnormality comprises:
calculating a critical value corresponding to each category according to the quantiles and the quantile distances corresponding to each category;
judging whether the electric power data to be analyzed of each category exceeds a corresponding critical value;
and marking the power data exceeding the critical value as abnormal power data.
5. The method of claim 1, further comprising:
storing the filtered power data in the database, and recording the storage time;
the reading of the last filtered power data from the database includes:
and reading the preset amount of power data with the latest storage time from the database.
6. The method according to claim 1, wherein the calculating the quantile and the quantile distance corresponding to the power data to be analyzed according to the power data filtered last time comprises:
arranging the power data which are filtered last time according to a sequence from small to large to obtain a data sequence;
segmenting the data sequence and determining the quantile corresponding to each segmentation point;
selecting two quantiles, and calculating the difference between the two quantiles to obtain the quantile distance.
7. The method of claim 6, wherein determining the quantile corresponding to each segmentation point comprises:
determining each segmentation point according to the data quantity contained in the data sequence, and calculating an integer part and a fraction part of each segmentation point;
and calculating quantiles corresponding to the segmentation points based on the integer part and the fraction part.
8. An electric power abnormal data filtering device based on a sub-bit distance algorithm is characterized by comprising:
the acquisition module is used for acquiring power data to be analyzed;
the reading module is used for reading the power data which is filtered last time from the database;
the calculation module is used for calculating the quantiles and the quantile distances corresponding to the power data to be analyzed according to the power data which is filtered last time;
and the filtering module is used for filtering the electric power data to be analyzed according to the quantiles and the quantile distances and marking the electric power data with abnormity.
9. An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to carry out the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010342686.8A CN111694819A (en) | 2020-04-27 | 2020-04-27 | Electric power abnormal data filtering method and device based on sub-bit distance algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010342686.8A CN111694819A (en) | 2020-04-27 | 2020-04-27 | Electric power abnormal data filtering method and device based on sub-bit distance algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111694819A true CN111694819A (en) | 2020-09-22 |
Family
ID=72476720
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010342686.8A Pending CN111694819A (en) | 2020-04-27 | 2020-04-27 | Electric power abnormal data filtering method and device based on sub-bit distance algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111694819A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830510A (en) * | 2018-07-16 | 2018-11-16 | 国网上海市电力公司 | A kind of electric power data preprocess method based on mathematical statistics |
| US20190007432A1 (en) * | 2017-06-29 | 2019-01-03 | Sap Se | Comparing unsupervised algorithms for anomaly detection |
| CN110426634A (en) * | 2019-09-10 | 2019-11-08 | 上海大制科技有限公司 | A kind of method and apparatus of the predicting abnormality for drive system |
-
2020
- 2020-04-27 CN CN202010342686.8A patent/CN111694819A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190007432A1 (en) * | 2017-06-29 | 2019-01-03 | Sap Se | Comparing unsupervised algorithms for anomaly detection |
| CN108830510A (en) * | 2018-07-16 | 2018-11-16 | 国网上海市电力公司 | A kind of electric power data preprocess method based on mathematical statistics |
| CN110426634A (en) * | 2019-09-10 | 2019-11-08 | 上海大制科技有限公司 | A kind of method and apparatus of the predicting abnormality for drive system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114298863B (en) | Data acquisition method and system of intelligent meter reading terminal | |
| CN110389269B (en) | Low-voltage distribution area topological relation identification method and device based on current optimization matching | |
| CN115170000A (en) | Remote monitoring method and system based on electric energy meter communication module | |
| CN109450089B (en) | Transformer area low voltage identification method and device and terminal equipment | |
| CN111694820A (en) | Electric power abnormal data multiple filtering method and device, electronic equipment and storage medium | |
| CN112016856B (en) | Comprehensive magnification abnormity identification method and device, metering system and storage medium | |
| CN111694821A (en) | Wiring mode-related power abnormal data filtering method and device and electronic equipment | |
| CN111694819A (en) | Electric power abnormal data filtering method and device based on sub-bit distance algorithm | |
| CN111880020A (en) | Fault recording data generation method and device for power distribution and utilization system of power consumer | |
| CN109064211A (en) | Sales service data analysing method, device and server | |
| CN115166625A (en) | Intelligent ammeter error estimation method and device | |
| CN115600831A (en) | User theoretical response potential evaluation method, device, terminal and storage medium | |
| CN112667567B (en) | Operation cost archiving method and device combining electric power data and power grid topology | |
| CN107194529B (en) | Power distribution network reliability economic benefit analysis method and device based on mining technology | |
| CN109507628B (en) | Reverse polarity fault monitoring method and device based on three-phase four-wire meter equipment | |
| Addy et al. | Understanding the effect of baseline modeling implementation choices on analysis of demand response performance | |
| CN112488532A (en) | Power equipment data monitoring method and device and server | |
| Alquthami et al. | Importance of smart meters data processing–case of saudi arabia | |
| CN112070535A (en) | Electric vehicle price prediction method and device | |
| Voloshko et al. | An improved pre-forecasting analysis of electrical loads of pumping station | |
| CN110807607A (en) | Low-voltage transformer area checking method based on big data platform and related device | |
| Kaszowska et al. | Assessment of Available Measurement Data, Data Breaks and Estimation of Missing Data from AMI Meters | |
| CN113435609B (en) | Line loss abnormity detection method and device and terminal equipment | |
| EP4050350A1 (en) | Determination of phase connections in a power grid | |
| CN115792369A (en) | Method and device for determining abnormal user metering point, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200922 |