CN112988725A - Power transmission line data cleaning method and system, electronic equipment and storage medium - Google Patents
Power transmission line data cleaning method and system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112988725A CN112988725A CN202110277177.6A CN202110277177A CN112988725A CN 112988725 A CN112988725 A CN 112988725A CN 202110277177 A CN202110277177 A CN 202110277177A CN 112988725 A CN112988725 A CN 112988725A
- Authority
- CN
- China
- Prior art keywords
- data
- value
- diagnosis
- trip
- repeated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 86
- 238000004140 cleaning Methods 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000003745 diagnosis Methods 0.000 claims abstract description 331
- 230000002159 abnormal effect Effects 0.000 claims abstract description 139
- 238000012217 deletion Methods 0.000 claims abstract description 11
- 230000037430 deletion Effects 0.000 claims abstract description 11
- 238000012549 training Methods 0.000 claims description 126
- 238000003062 neural network model Methods 0.000 claims description 45
- 238000012545 processing Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012216 screening Methods 0.000 abstract description 5
- 230000015654 memory Effects 0.000 description 14
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000007812 deficiency Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001681 protective effect Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001012 protector Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Remote Monitoring And Control Of Power-Distribution Networks (AREA)
Abstract
The embodiment of the invention provides a method and a system for cleaning transmission line data, electronic equipment and a storage medium, wherein the method for cleaning the transmission line data comprises the following steps: acquiring data of the power transmission line; dividing the power transmission line data into fault diagnosis data, protection action data and trip data; inputting each type of data into a corresponding deletion value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain deletion values, abnormal values and repeated values of each type of data; and correcting the missing value, the abnormal value and the repeated value of each type of data according to the characteristics of the corresponding data type to obtain the cleaned data. According to the method and the device, the data type division and the dirty data type division are combined, so that the dirty data screening is more targeted, and the cleaning accuracy is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a method and a system for cleaning transmission line data, electronic equipment and a storage medium.
Background
In order to ensure the normal operation of the power transmission line, various monitoring devices are installed in the power transmission line and used for acquiring the operation parameters of the power transmission line, so that the operation condition of the power transmission line can be known in real time. However, due to communication abnormality, equipment failure and other reasons, the collected data is likely to have more abnormal data, repeated data and missing data, thereby affecting the judgment of the operating condition of the power transmission line. Therefore, a data cleansing step is typically required to remove dirty data prior to processing the acquired data.
The existing data cleaning method generally collects various data and then uniformly cleans the data, and the cleaning method has no pertinence and low cleaning accuracy.
Disclosure of Invention
The embodiment of the invention provides a method and a system for cleaning transmission line data, electronic equipment and a storage medium, which are used for ensuring that screening of dirty data is more targeted and improving the cleaning accuracy.
In a first aspect, an embodiment of the present invention provides a method for cleaning data of a power transmission line, including:
acquiring data of the power transmission line;
dividing the power transmission line data into fault diagnosis data, protection action data and trip data;
inputting each type of data into a corresponding deletion value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain deletion values, abnormal values and repeated values of each type of data;
and correcting the missing value, the abnormal value and the repeated value of each type of data according to the characteristics of the corresponding data type to obtain the cleaned data.
Optionally, the dividing the power transmission line data into fault diagnosis data, protection action data, and trip data specifically includes:
and dividing the power transmission line data into fault diagnosis data, protection action data and trip data according to the equipment and the position of the acquired data.
Optionally, the inputting each type of data into the corresponding deficiency value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain the deficiency value, abnormal value and repeated value of each type of data specifically includes:
inputting the fault diagnosis data into a fault diagnosis missing value diagnosis model, a fault diagnosis abnormal value diagnosis model and a fault diagnosis repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the fault diagnosis data;
respectively inputting the protection action data into a protection action missing value diagnosis model, a protection action abnormal value diagnosis model and a protection action repeated value diagnosis model to obtain a missing value, an abnormal value and a repeated value of the protection action data;
and respectively inputting the trip data into a trip missing value diagnosis model, a trip abnormal value diagnosis model and a trip repeated value diagnosis model to obtain the missing value, the abnormal value and the repeated value of the trip data.
Optionally, the training mode of the fault diagnosis missing value diagnosis model is as follows: inputting the fault diagnosis historical data and missing values in the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis missing value diagnosis model;
the training mode of the fault diagnosis abnormal value diagnosis model is as follows: inputting abnormal values in the fault diagnosis historical data and the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis abnormal value diagnosis model;
the training mode of the fault diagnosis repeated value diagnosis model is as follows: inputting the repeated values in the fault diagnosis historical data and the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis repeated value diagnosis model;
the training mode of the protection action missing value diagnosis model is as follows: inputting the protection action type historical data and the missing value in the protection action type historical data into a neural network model as training samples to be trained to obtain a protection action missing value diagnosis model;
the training mode of the protection action abnormal value diagnosis model is as follows: inputting abnormal values in the protection action type historical data and the protection action type historical data into a neural network model as training samples to be trained to obtain a protection action abnormal value diagnosis model;
the training mode of the protection action repetition value diagnosis model is as follows: inputting the protection action type historical data and the repeated value in the protection action type historical data as training samples into a neural network model for training to obtain a protection action repeated value diagnosis model;
the training mode of the tripping failure value diagnosis model is as follows: inputting the trip type historical data and the missing value in the trip type historical data as training samples into a neural network model for training to obtain a trip missing value diagnosis model;
the method for training the trip abnormal value diagnosis model comprises the following steps: inputting the trip type historical data and abnormal values in the trip type historical data into a neural network model as training samples to be trained to obtain a trip abnormal value diagnosis model;
the trip repetition value diagnosis model is trained in the following way: and inputting the trip type historical data and the repeated value in the trip type historical data as training samples into a neural network model for training to obtain the trip repeated value diagnosis model.
Optionally, the modifying missing values, abnormal values, and repeated values of each type of data according to characteristics of the corresponding data type to obtain the cleaned data specifically includes:
processing missing values of various data by using an interpolation method or an interpolation method;
deleting or replacing the abnormal values of various data by average values;
and deleting the repeated values of various types of data.
In a second aspect, an embodiment of the present invention provides a system for cleaning data of a power transmission line, including:
the data acquisition module is used for acquiring the data of the power transmission line;
the data dividing module is used for dividing the power transmission line data into fault diagnosis data, protection action data and trip data;
the diagnosis module is used for inputting each type of data into the corresponding deletion value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain the deletion value, abnormal value and repeated value of each type of data;
and the cleaning module is used for correcting the missing value, the abnormal value and the repeated value of each type of data according to the characteristics of the corresponding data type to obtain the cleaned data.
Optionally, the data dividing module specifically includes:
and the dividing unit is used for dividing the power transmission line data into fault diagnosis data, protection action data and trip data according to the equipment and the position of the acquired data.
Optionally, the diagnostic module specifically includes:
the fault diagnosis data diagnosis unit is used for inputting the fault diagnosis data into a fault diagnosis missing value diagnosis model, a fault diagnosis abnormal value diagnosis model and a fault diagnosis repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the fault diagnosis data;
the protection action type data diagnosis unit is used for inputting the protection action type data into a protection action missing value diagnosis model, a protection action abnormal value diagnosis model and a protection action repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the protection action type data;
and the trip data diagnosis unit is used for inputting the trip data into the trip missing value diagnosis model, the trip abnormal value diagnosis model and the trip repeated value diagnosis model respectively to obtain the missing value, the abnormal value and the repeated value of the trip data.
Optionally, the power transmission line data cleaning system further includes a training module; the training module specifically comprises:
the first training unit is used for inputting the fault diagnosis historical data and missing values in the fault diagnosis historical data into a neural network model as training samples to be trained to obtain a fault diagnosis missing value diagnosis model;
the second training unit is used for inputting abnormal values in the fault diagnosis historical data and the fault diagnosis historical data into a neural network model as training samples to be trained to obtain the fault diagnosis abnormal value diagnosis model;
a third training unit, configured to input a repeated value in the fault diagnosis-type historical data and the fault diagnosis-type historical data as a training sample into a neural network model for training to obtain a fault diagnosis repeated value diagnosis model;
the fourth training unit is used for inputting the protection action type historical data and the missing value in the protection action type historical data into a neural network model as training samples to be trained to obtain the protection action missing value diagnosis model;
a fifth training unit, configured to input abnormal values in the protection action type historical data and the protection action type historical data as training samples into a neural network model for training to obtain the protection action abnormal value diagnosis model;
a sixth training unit, configured to input a repetition value in the protection action type historical data and the protection action type historical data as a training sample into a neural network model for training to obtain the protection action repetition value diagnostic model;
a seventh training unit, configured to input the trip-type historical data and a missing value in the trip-type historical data as training samples into a neural network model for training to obtain the trip missing value diagnosis model;
the eighth training unit is used for inputting abnormal values in the trip type historical data and the trip type historical data into a neural network model as training samples to be trained to obtain the trip abnormal value diagnosis model;
and the ninth training unit is used for inputting the trip type historical data and the repeated value in the trip type historical data into a neural network model as a training sample to be trained so as to obtain the trip repeated value diagnosis model.
Optionally, the cleaning module specifically includes:
the first cleaning unit is used for processing missing values of various data by utilizing an interpolation method or an interpolation method;
the second cleaning unit is used for deleting or replacing the average value of the abnormal values of the various types of data;
and the third cleaning unit is used for deleting the repeated values of various types of data.
In a third aspect, an embodiment of the present invention provides an electronic device, where the electronic device includes:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the method for cleaning the power transmission line data according to the first aspect of the present invention.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for cleaning transmission line data according to the first aspect of the present invention.
After the power transmission line data are obtained, the power transmission line data are divided into fault diagnosis data, protection action data and trip data, each type of data are input into a corresponding missing value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain missing values, abnormal values and repeated values of each type of data, and the missing values, abnormal values and repeated values of each type of data are corrected according to the characteristics of the corresponding data type to obtain the cleaned data. The data are divided into fault diagnosis data, protection action data and trip data, the data are input into corresponding diagnosis models to determine missing values, abnormal values and repetition of the data, and the data type division and the dirty data type division are combined, so that the dirty data screening is more targeted, and the cleaning accuracy is improved.
Drawings
Fig. 1 is a flowchart of a method for cleaning transmission line data according to a first embodiment of the present invention;
fig. 2 is a flowchart of a method for cleaning data of a power transmission line according to a second embodiment of the present invention;
fig. 3 is a structural diagram of a power transmission line data cleaning system according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures. The embodiments and features of the embodiments in the present application may be combined with each other without conflict.
Example one
Fig. 1 is a flowchart of steps of a method for cleaning transmission line data according to an embodiment of the present invention, where the method is applicable to a case of cleaning transmission line data before analyzing the transmission line data, and the method may be executed by a system for cleaning transmission line data according to an embodiment of the present invention, and the system for cleaning transmission line data may be implemented by hardware or software and integrated in an electronic device according to an embodiment of the present invention, and specifically, as shown in fig. 1, the method for cleaning transmission line data according to an embodiment of the present invention may include the following steps:
and S101, acquiring the data of the power transmission line.
In the embodiment of the present invention, the transmission line data may be data reported by data acquisition equipment on the transmission line, and the data acquisition equipment may include electrical parameter acquisition equipment, such as electrical parameters acquired by current, voltage, and power acquisition equipment, and may also include action data of automation equipment on the transmission line, such as action data of protection equipment, or further include fault data acquired by fault diagnosis equipment.
S102, dividing the power transmission line data into fault diagnosis data, protection action data and trip data.
In an optional embodiment, the fault diagnosis data at least includes data collected by the fault diagnosis device and data collected by other devices and capable of reflecting the fault of the power transmission line, the protection action data at least includes data such as whether the protection device (e.g., an overvoltage/overcurrent protector) acts, action time, and collected electrical signals, the trip data at least includes data such as a number of the trip device, a position of the trip device, and a line where the trip device is located, and the power transmission line data can be divided into the fault diagnosis data, the protection action data, and the trip data according to the device (including a device number, a device action condition, data collected by the device, and the like) and the position (including a geographical position and a line location and the like) of the collected data.
S103, inputting each type of data into the corresponding missing value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain the missing value, abnormal value and repeated value of each type of data.
In the embodiment of the present invention, a missing value diagnosis model may be trained in advance to diagnose whether missing values exist in data, an abnormal value diagnosis model may be trained to diagnose whether abnormal values exist in data, and a repeated value diagnosis model may be trained to diagnose whether data are repeated.
For each type of divided data, each type of data can be respectively input into the missing value diagnosis model, the abnormal value diagnosis model and the repeated value diagnosis model to obtain the missing value, the abnormal value and the repeated value of each type of data.
And S104, correcting the missing value, the abnormal value and the repeated value of each type of data according to the characteristics of the corresponding data type to obtain the cleaned data.
The missing value is data that should be acquired but not acquired, and the abnormal value is data whose value is out of the allowable range, for example, it is usually specified that the device is operated, i.e., marked as 1, and is not operated, i.e., marked as 0, and belongs to the abnormal value if the received data is neither 1 nor 0, and the repeated value is data that is received at least twice at the same time.
Illustratively, missing values can be supplemented by interpolation calculations, outliers can be deleted or replaced by averages, and redundant data can be deleted for duplicates.
After the power transmission line data are obtained, the power transmission line data are divided into fault diagnosis data, protection action data and trip data, each type of data are input into a corresponding missing value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain missing values, abnormal values and repeated values of each type of data, and the missing values, abnormal values and repeated values of each type of data are corrected according to the characteristics of the corresponding data type to obtain the cleaned data. The data are divided into fault diagnosis data, protection action data and trip data, the data are input into corresponding diagnosis models to determine missing values, abnormal values and repetition of the data, and the data type division and the dirty data type division are combined, so that the dirty data screening is more targeted, and the cleaning accuracy is improved.
Example two
Fig. 2 is a flowchart of steps of a method for cleaning transmission line data according to a second embodiment of the present invention, where the second embodiment of the present invention is optimized based on the first embodiment, and as shown in fig. 2, the method for cleaning transmission line data according to the second embodiment of the present invention may include the following steps:
s201, acquiring the data of the power transmission line.
S202, dividing the power transmission line data into fault diagnosis data, protection action data and trip data according to the data acquisition equipment and the data acquisition positions.
S203, inputting the fault diagnosis data into a fault diagnosis missing value diagnosis model, a fault diagnosis abnormal value diagnosis model and a fault diagnosis repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the fault diagnosis data.
In the embodiment of the present invention, the fault diagnosis missing value diagnosis model may be trained in advance by the following means: inputting the fault diagnosis historical data and the missing values in the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis missing value diagnosis model, and training a fault diagnosis abnormal value diagnosis model in the following modes: inputting abnormal values in the fault diagnosis historical data and the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis abnormal value diagnosis model; and training a fault diagnosis repetitive value diagnosis model by: and inputting the repeated values in the fault diagnosis historical data and the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis repeated value diagnosis model.
The fault diagnosis missing value diagnosis model, the fault diagnosis missing value diagnosis model and the fault diagnosis repeated value diagnosis model can be neural networks such as CNN, RNN and DNN, and the specific training mode can refer to a common neural network supervised training mode.
For the fault diagnosis data, the fault diagnosis data can be respectively input into a fault diagnosis missing value diagnosis model, a fault diagnosis abnormal value diagnosis model and a fault diagnosis repeated value diagnosis model which are trained in advance, so as to obtain a missing value, an abnormal value and a repeated value of the fault diagnosis data.
And S204, inputting the protection action data into a protection action missing value diagnosis model, a protection action abnormal value diagnosis model and a protection action repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the protection action data.
Specifically, in the embodiment of the present invention, the training mode of the protection action deficiency value diagnosis model is as follows: inputting the protection action type historical data and the missing value in the protection action type historical data into a neural network model as training samples for training to obtain a protection action missing value diagnosis model; the training mode of the protective action abnormal value diagnosis model is as follows: inputting abnormal values in the protection action type historical data and the protection action type historical data into a neural network model as training samples for training to obtain a protection action abnormal value diagnosis model; the training mode of the protective action repeated value diagnosis model is as follows: and inputting the protection action type historical data and the repeated value in the protection action type historical data as training samples into a neural network model for training to obtain a protection action repeated value diagnosis model.
The specific training mode of the diagnostic model can refer to a common neural network supervised training mode, and the training mode is not detailed in the embodiment of the invention.
For the protection action data, the protection action data can be respectively input into a fault diagnosis missing value diagnosis model, a fault diagnosis abnormal value diagnosis model and a fault diagnosis repeated value diagnosis model which are trained in advance, so that a missing value, an abnormal value and a repeated value of the protection action data can be obtained.
And S205, inputting the trip data into a trip missing value diagnosis model, a trip abnormal value diagnosis model and a trip repeated value diagnosis model respectively to obtain the missing value, the abnormal value and the repeated value of the trip data.
In an optional embodiment of the present invention, the trip missing value diagnosis model is trained by: and inputting the trip type historical data and the missing values in the trip type historical data as training samples into a neural network model for training to obtain a trip missing value diagnosis model. The method for training the trip abnormal value diagnosis model comprises the following steps: and inputting abnormal values in the trip type historical data and the trip type historical data as training samples into a neural network model for training to obtain a trip abnormal value diagnosis model. The trip repetition value diagnosis model is trained in the following way: and inputting the trip type historical data and the repeated value in the trip type historical data as training samples into a neural network model for training to obtain a trip repeated value diagnosis model.
The specific training mode of the diagnostic model can refer to a common neural network supervised training mode, and the training mode is not detailed in the embodiment of the invention.
For the trip data, the trip data can be respectively input into a fault diagnosis missing value diagnosis model, a fault diagnosis abnormal value diagnosis model and a fault diagnosis repeated value diagnosis model which are trained in advance, so as to obtain a missing value, an abnormal value and a repeated value of the trip data.
And S206, processing missing values of various data by utilizing an interpolation method or an interpolation method.
The interpolation means that values between 2 or more values are calculated by using a certain function, for example, a commonly used mean linear interpolation may be used for each minute of current value on a power line, for example, on one power line, the current I1 at the I +1 th minute is 10A, the current I2 at the I +2 th minute is 9.8A, the current I3 at the I +3 th minute is 9.9A, the current at the I +4 th minute is not collected, the current I5 at the I +5 th minute is 10.1A, the current I7 at the I +6 th minute is 9.7A, the current I4 at the I +4 th minute is interpolated (I3+ I5)/2 is 10A, or the current I4 at the I +4 th minute is interpolated (I2+ I3+ I5+ I6)/4 is 9.88A, or a linear interpolation, a segmentation diagnosis data such as a newton-class, a segmentation method, and a method for a fault diagnosis data, And interpolating missing values in the protection action data and the trip data to obtain complete fault diagnosis data, protection action data and trip data.
And S207, deleting or replacing the abnormal values of the various types of data by the average value.
In the embodiment of the present invention, the abnormal value may refer to a value out of a preset range, such as a current value greater than a preset current value, a voltage value greater than a preset voltage value, and the like. The abnormal value can be deleted or replaced by a sampling average value after deletion, and a plurality of numerical values near the abnormal value can be selected according to a preset window to carry out smooth processing on the abnormal value, so that accurate fault diagnosis data, protection action data and trip data can be obtained.
Illustratively, for a set of voltage values: 10.1KV, 13KV, 10KV and 10.2KV, if the power transmission line is a 10KV power transmission line, and the 13KV obviously belongs to an abnormal value, the 13KV can be deleted and the average value of the 10.1KV and the 10KV can be calculated to replace the 13KV, or the average value of the 10.1KV, the 10KV and the 10.2KV can be calculated to replace the 10.1KV and the 10 KV.
And S208, deleting the repeated values of the various types of data.
Specifically, the repetition value may refer to data recorded in the same record, and exemplarily refer to data repeatedly reported by the same acquisition device, for example, the repetition value may be two identical current values reported by the acquisition device a twice at a certain time by the node B. The fault diagnosis data, the protection action data and the trip data can be subjected to duplicate removal processing to obtain the duplicate-removed fault diagnosis data, protection action data and trip data.
It should be noted that, in the embodiment of the present invention, there is no limitation on the correction sequence of the missing value, the abnormal value, and the repeated value of the data, in a preferred embodiment, the deduplication processing may be performed first, then the abnormal value correction may be performed, and finally the missing value interpolation may be performed, so as to obtain the complete and accurate fault diagnosis data, protection action data, and trip data after cleaning, and the fault diagnosis may be performed on the power transmission line by using the fault diagnosis data, protection action data, and trip data after cleaning, so as to improve the accuracy of fault diagnosis.
After the power transmission line data are obtained, the power transmission line data are divided into fault diagnosis data, protection action data and trip data, each type of data are input into a corresponding missing value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain missing values, abnormal values and repeated values of each type of data, and the missing values, abnormal values and repeated values of each type of data are corrected according to the characteristics of the corresponding data type to obtain the cleaned data. The data are divided into fault diagnosis data, protection action data and trip data, the data are input into corresponding diagnosis models to determine missing values, abnormal values and repetition of the data, and the data type division and the dirty data type division are combined, so that the dirty data screening is more targeted, and the cleaning accuracy is improved.
EXAMPLE III
Fig. 3 is a block diagram of a power transmission line data cleaning system according to a third embodiment of the present invention, and as shown in fig. 3, the power transmission line data cleaning system according to the third embodiment of the present invention may specifically include the following modules:
and the data acquisition module 301 is configured to acquire the data of the power transmission line.
The data dividing module 302 is configured to divide the power transmission line data into fault diagnosis class data, protection action class data, and trip class data.
The diagnosis module 303 is configured to input each type of data into the corresponding deficiency value diagnosis model, abnormal value diagnosis model, and repeated value diagnosis model, so as to obtain a deficiency value, an abnormal value, and a repeated value of each type of data.
And the cleaning module 304 is configured to correct the missing values, the abnormal values, and the repeated values of each type of data according to the characteristics of the corresponding data type, so as to obtain cleaned data.
As an optional implementation manner, the data partitioning module 302 specifically includes:
and the dividing unit is used for dividing the power transmission line data into fault diagnosis data, protection action data and trip data according to the equipment and the position of the acquired data.
As an optional implementation manner, the diagnosis module 303 specifically includes:
and the fault diagnosis data diagnosis unit is used for inputting the fault diagnosis data into the fault diagnosis missing value diagnosis model, the fault diagnosis abnormal value diagnosis model and the fault diagnosis repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the fault diagnosis data.
And the protection action data diagnosis unit is used for inputting the protection action data into a protection action missing value diagnosis model, a protection action abnormal value diagnosis model and a protection action repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the protection action data.
And the trip data diagnosis unit is used for inputting the trip data into the trip missing value diagnosis model, the trip abnormal value diagnosis model and the trip repeated value diagnosis model respectively to obtain the missing value, the abnormal value and the repeated value of the trip data.
As an optional implementation manner, the power transmission line data cleaning system further includes a training module; the training module specifically comprises:
and the first training unit is used for inputting the fault diagnosis historical data and the missing values in the fault diagnosis historical data into a neural network model as training samples to be trained so as to obtain the fault diagnosis missing value diagnosis model.
And the second training unit is used for inputting abnormal values in the fault diagnosis historical data and the fault diagnosis historical data as training samples into a neural network model for training to obtain the fault diagnosis abnormal value diagnosis model.
And the third training unit is used for inputting the repeated values in the fault diagnosis historical data and the fault diagnosis historical data as training samples into a neural network model for training to obtain the fault diagnosis repeated value diagnosis model.
And the fourth training unit is used for inputting the protection action type historical data and the missing values in the protection action type historical data into a neural network model as training samples to be trained to obtain the protection action missing value diagnosis model.
And the fifth training unit is used for inputting abnormal values in the protection action type historical data and the protection action type historical data into a neural network model as training samples to be trained to obtain the protection action abnormal value diagnosis model.
And the sixth training unit is used for inputting the protection action type historical data and the repeated value in the protection action type historical data as training samples into a neural network model for training to obtain the protection action repeated value diagnosis model.
And the seventh training unit is used for inputting the trip type historical data and the missing values in the trip type historical data into a neural network model as training samples to be trained so as to obtain the trip missing value diagnosis model.
And the eighth training unit is used for inputting abnormal values in the trip type historical data and the trip type historical data as training samples into a neural network model for training to obtain the trip abnormal value diagnosis model.
And the ninth training unit is used for inputting the trip type historical data and the repeated value in the trip type historical data into a neural network model as a training sample to be trained so as to obtain the trip repeated value diagnosis model.
As an optional implementation manner, the cleaning module 304 specifically includes:
and the first cleaning unit is used for processing the missing values of various types of data by utilizing an interpolation method or an interpolation method.
And the second cleaning unit is used for deleting or replacing the average value of the abnormal values of the various types of data.
And the third cleaning unit is used for deleting the repeated values of various types of data.
The power transmission line data cleaning system provided by the embodiment of the invention can execute the power transmission line data cleaning method provided by the first embodiment and the second embodiment of the invention, and has corresponding functions and beneficial effects of the execution method.
Example four
Referring to fig. 4, a schematic structural diagram of an electronic device in one example of the invention is shown. As shown in fig. 4, the electronic device may specifically include: a processor 401, a memory 402, a display screen 403 with touch functionality, an input device 404, an output device 405, and a communication device 406. The number of the processors 401 in the electronic device may be one or more, and one processor 401 is taken as an example in fig. 4. The number of the memories 402 in the electronic device may be one or more, and one memory 402 is taken as an example in fig. 4. The processor 401, the memory 402, the display 403, the input means 404, the output means 405 and the communication means 406 of the device may be connected by a bus or other means, as exemplified by the bus connection in fig. 4.
The memory 402 is used as a computer-readable storage medium and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the power transmission line data cleaning method according to any embodiment of the present invention (for example, the data acquisition module 301, the data partitioning module 302, the diagnosis module 303, and the cleaning module 304 in the power transmission line data cleaning system), and the memory 402 mainly includes a storage program area and a storage data area, where the storage program area can store an operating device and an application program required by at least one function; the storage data area may store data created according to use of the device, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 402 may further include memory located remotely from the processor 401, which may be connected to the device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The display screen 403 is a display screen 403 with a touch function, which may be a capacitive screen, an electromagnetic screen, or an infrared screen. In general, the display screen 403 is used for displaying data according to instructions of the processor 401, and is also used for receiving touch operations applied to the display screen 403 and sending corresponding signals to the processor 401 or other devices. Optionally, when the display screen 403 is an infrared screen, the display screen further includes an infrared touch frame, and the infrared touch frame is disposed around the display screen 403, and may also be configured to receive an infrared signal and send the infrared signal to the processor 401 or other devices.
The communication device 406 is used for establishing a communication connection with other devices, and may be a wired communication device and/or a wireless communication device.
The input device 404 may be used to receive input numeric or character information and generate key signal inputs relating to user settings and function controls of the apparatus. The output device 405 may include an audio device such as a speaker. It should be noted that the specific composition of the input device 404 and the output device 405 may be set according to actual conditions.
The processor 401 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 402, so as to implement the above-mentioned power transmission line data cleaning method.
Specifically, in the embodiment, when the processor 401 executes one or more programs stored in the memory 402, the steps of the method for cleaning the power transmission line data provided in the embodiment of the present invention are specifically implemented.
EXAMPLE five
Fifth, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, may implement the method for cleaning transmission line data in any embodiment of the present invention.
Of course, the storage medium containing the computer-executable instructions provided by the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the method for cleaning the power transmission line data provided by any embodiment of the present invention.
It should be noted that, as for the system, the electronic device, and the storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and in relevant places, reference may be made to the partial description of the method embodiments.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, where the computer software product may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk, or an optical disk of a computer, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) to execute the method for cleaning power transmission line data according to the embodiments of the present invention.
It should be noted that, in the embodiment of the power transmission line data cleaning system, each included unit and module are only divided according to functional logic, but are not limited to the above division, as long as corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for cleaning transmission line data is characterized by comprising the following steps:
acquiring data of the power transmission line;
dividing the power transmission line data into fault diagnosis data, protection action data and trip data;
inputting each type of data into a corresponding deletion value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain deletion values, abnormal values and repeated values of each type of data;
and correcting the missing value, the abnormal value and the repeated value of each type of data according to the characteristics of the corresponding data type to obtain the cleaned data.
2. The method for cleaning the data of the power transmission line according to claim 1, wherein the dividing of the data of the power transmission line into fault diagnosis class data, protection action class data and trip class data specifically comprises:
and dividing the power transmission line data into fault diagnosis data, protection action data and trip data according to the equipment and the position of the acquired data.
3. The method for cleaning the data of the power transmission line according to claim 1, wherein each type of data is input into a corresponding missing value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain missing values, abnormal values and repeated values of each type of data, and the method specifically comprises the following steps:
inputting the fault diagnosis data into a fault diagnosis missing value diagnosis model, a fault diagnosis abnormal value diagnosis model and a fault diagnosis repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the fault diagnosis data;
respectively inputting the protection action data into a protection action missing value diagnosis model, a protection action abnormal value diagnosis model and a protection action repeated value diagnosis model to obtain a missing value, an abnormal value and a repeated value of the protection action data;
and respectively inputting the trip data into a trip missing value diagnosis model, a trip abnormal value diagnosis model and a trip repeated value diagnosis model to obtain the missing value, the abnormal value and the repeated value of the trip data.
4. The method according to claim 3,
the training mode of the fault diagnosis missing value diagnosis model is as follows: inputting the fault diagnosis historical data and missing values in the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis missing value diagnosis model;
the training mode of the fault diagnosis abnormal value diagnosis model is as follows: inputting abnormal values in the fault diagnosis historical data and the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis abnormal value diagnosis model;
the training mode of the fault diagnosis repeated value diagnosis model is as follows: inputting the repeated values in the fault diagnosis historical data and the fault diagnosis historical data as training samples into a neural network model for training to obtain a fault diagnosis repeated value diagnosis model;
the training mode of the protection action missing value diagnosis model is as follows: inputting the protection action type historical data and the missing value in the protection action type historical data into a neural network model as training samples to be trained to obtain a protection action missing value diagnosis model;
the training mode of the protection action abnormal value diagnosis model is as follows: inputting abnormal values in the protection action type historical data and the protection action type historical data into a neural network model as training samples to be trained to obtain a protection action abnormal value diagnosis model;
the training mode of the protection action repetition value diagnosis model is as follows: inputting the protection action type historical data and the repeated value in the protection action type historical data as training samples into a neural network model for training to obtain a protection action repeated value diagnosis model;
the training mode of the tripping failure value diagnosis model is as follows: inputting the trip type historical data and the missing value in the trip type historical data as training samples into a neural network model for training to obtain a trip missing value diagnosis model;
the method for training the trip abnormal value diagnosis model comprises the following steps: inputting the trip type historical data and abnormal values in the trip type historical data into a neural network model as training samples to be trained to obtain a trip abnormal value diagnosis model;
the trip repetition value diagnosis model is trained in the following way: and inputting the trip type historical data and the repeated value in the trip type historical data as training samples into a neural network model for training to obtain the trip repeated value diagnosis model.
5. The method for cleaning the data of the power transmission line according to claim 1, wherein the method for correcting the missing value, the abnormal value and the repeated value of each type of data according to the characteristics of the corresponding data type to obtain the cleaned data specifically comprises the following steps:
processing missing values of various data by using an interpolation method or an interpolation method;
deleting or replacing the abnormal values of various data by average values;
and deleting the repeated values of various types of data.
6. A transmission line data cleaning system, comprising:
the data acquisition module is used for acquiring the data of the power transmission line;
the data dividing module is used for dividing the power transmission line data into fault diagnosis data, protection action data and trip data;
the diagnosis module is used for inputting each type of data into the corresponding deletion value diagnosis model, abnormal value diagnosis model and repeated value diagnosis model to obtain the deletion value, abnormal value and repeated value of each type of data;
and the cleaning module is used for correcting the missing value, the abnormal value and the repeated value of each type of data according to the characteristics of the corresponding data type to obtain the cleaned data.
7. The system for cleaning the data of the power transmission line according to claim 6, wherein the data dividing module specifically comprises:
and the dividing unit is used for dividing the power transmission line data into fault diagnosis data, protection action data and trip data according to the equipment and the position of the acquired data.
8. The system for cleaning data of power transmission lines according to claim 6, wherein the diagnostic module specifically comprises:
the fault diagnosis data diagnosis unit is used for inputting the fault diagnosis data into a fault diagnosis missing value diagnosis model, a fault diagnosis abnormal value diagnosis model and a fault diagnosis repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the fault diagnosis data;
the protection action type data diagnosis unit is used for inputting the protection action type data into a protection action missing value diagnosis model, a protection action abnormal value diagnosis model and a protection action repeated value diagnosis model respectively to obtain a missing value, an abnormal value and a repeated value of the protection action type data;
and the trip data diagnosis unit is used for inputting the trip data into the trip missing value diagnosis model, the trip abnormal value diagnosis model and the trip repeated value diagnosis model respectively to obtain the missing value, the abnormal value and the repeated value of the trip data.
9. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of power transmission line data cleansing of any of claims 1-5.
10. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method for cleaning transmission line data according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110277177.6A CN112988725A (en) | 2021-03-15 | 2021-03-15 | Power transmission line data cleaning method and system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110277177.6A CN112988725A (en) | 2021-03-15 | 2021-03-15 | Power transmission line data cleaning method and system, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112988725A true CN112988725A (en) | 2021-06-18 |
Family
ID=76336484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110277177.6A Pending CN112988725A (en) | 2021-03-15 | 2021-03-15 | Power transmission line data cleaning method and system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988725A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113760880A (en) * | 2021-09-07 | 2021-12-07 | 天津大学 | Pretreatment method of water quality automatic monitoring data |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509982A (en) * | 2018-03-12 | 2018-09-07 | 昆明理工大学 | A method of the uneven medical data of two classification of processing |
| CN109102103A (en) * | 2018-06-26 | 2018-12-28 | 上海鲁班软件股份有限公司 | A kind of multi-class energy consumption prediction technique based on Recognition with Recurrent Neural Network |
| CN109299156A (en) * | 2018-08-21 | 2019-02-01 | 平安科技（深圳）有限公司 | Electronic device, the electric power data predicting abnormality method based on XGBoost and storage medium |
| CN109886306A (en) * | 2019-01-24 | 2019-06-14 | 国网山东省电力公司德州供电公司 | A kind of electric network failure diagnosis data cleaning method |
| CN111582593A (en) * | 2020-05-13 | 2020-08-25 | 山东博依特智能科技有限公司 | Data cleaning integration method based on industrial power consumption big data |
| US20210034994A1 (en) * | 2019-08-02 | 2021-02-04 | Capital One Services, Llc | Computer-based systems configured for detecting, classifying, and visualizing events in large-scale, multivariate and multidimensional datasets and methods of use thereof |
| CN112346941A (en) * | 2019-08-08 | 2021-02-09 | 北京国双科技有限公司 | Fault diagnosis method and device |
-
2021
- 2021-03-15 CN CN202110277177.6A patent/CN112988725A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509982A (en) * | 2018-03-12 | 2018-09-07 | 昆明理工大学 | A method of the uneven medical data of two classification of processing |
| CN109102103A (en) * | 2018-06-26 | 2018-12-28 | 上海鲁班软件股份有限公司 | A kind of multi-class energy consumption prediction technique based on Recognition with Recurrent Neural Network |
| CN109299156A (en) * | 2018-08-21 | 2019-02-01 | 平安科技（深圳）有限公司 | Electronic device, the electric power data predicting abnormality method based on XGBoost and storage medium |
| CN109886306A (en) * | 2019-01-24 | 2019-06-14 | 国网山东省电力公司德州供电公司 | A kind of electric network failure diagnosis data cleaning method |
| US20210034994A1 (en) * | 2019-08-02 | 2021-02-04 | Capital One Services, Llc | Computer-based systems configured for detecting, classifying, and visualizing events in large-scale, multivariate and multidimensional datasets and methods of use thereof |
| CN112346941A (en) * | 2019-08-08 | 2021-02-09 | 北京国双科技有限公司 | Fault diagnosis method and device |
| CN111582593A (en) * | 2020-05-13 | 2020-08-25 | 山东博依特智能科技有限公司 | Data cleaning integration method based on industrial power consumption big data |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113760880A (en) * | 2021-09-07 | 2021-12-07 | 天津大学 | Pretreatment method of water quality automatic monitoring data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110601898B (en) | Abnormity early warning method, abnormity early warning device, server and storage medium | |
| CN111143102B (en) | Abnormal data detection method and device, storage medium and electronic equipment | |
| CN110830450A (en) | Abnormal flow monitoring method, device and equipment based on statistics and storage medium | |
| CN106685676B (en) | Node switching method and device | |
| EP3896420A1 (en) | Method, device and electronic apparatus for detecting performance status of optical module in engineering application | |
| CN111444067A (en) | Rule engine-based configuration system monitoring method, device and equipment | |
| CN111049571B (en) | Optical module fault prediction method and device and computer readable storage medium | |
| CN110824304A (en) | Method for analyzing insulation degradation trend of zinc oxide arrester | |
| CN114567538A (en) | Alarm information processing method and device | |
| CN112988725A (en) | Power transmission line data cleaning method and system, electronic equipment and storage medium | |
| CN107450041B (en) | Current sensor fault diagnosis method and device for synchronous motor and frequency converter | |
| CN115935825A (en) | Power distribution network fault processing method and device based on digital twin model | |
| CN113487182B (en) | Device health state evaluation method, device, computer device and medium | |
| US11657148B2 (en) | Event analysis in an electric power system | |
| US20210255912A1 (en) | Determination of a reliability state of an electrical network | |
| CN112730654A (en) | Fault detection method and device for sulfur hexafluoride electrical equipment and terminal equipment | |
| CN113033889B (en) | High-voltage transmission line fault prediction method and device and terminal equipment | |
| CN111245937A (en) | Comprehensive monitoring remote inspection system | |
| CN110927552A (en) | On-line monitoring VCE_ONFailure prediction method and device for IGBT power module of auxiliary inverter | |
| JP2016039610A (en) | Data transmitter and remote monitoring system | |
| CN114358121A (en) | Monitoring method and device based on substation equipment and terminal equipment | |
| CN109638845B (en) | Method and device for establishing switching combination scheme of power transmission line | |
| CN114253800A (en) | Edge diagnosis method, device, equipment and storage medium | |
| Chiodo et al. | Bayes inference for reliability of HV insulation systems in the presence of switching voltage surges using a Weibull stress-strength model | |
| CN113095952A (en) | Power grid splitting early warning method and system considering real-time and N-X forecast information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210618 |
|
| RJ01 | Rejection of invention patent application after publication |