CN115905341A - Data quality abnormity detection method and device, electronic equipment and storage medium - Google Patents
Data quality abnormity detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115905341A CN115905341A CN202111153105.7A CN202111153105A CN115905341A CN 115905341 A CN115905341 A CN 115905341A CN 202111153105 A CN202111153105 A CN 202111153105A CN 115905341 A CN115905341 A CN 115905341A
- Authority
- CN
- China
- Prior art keywords
- sensor
- sensor signal
- signal
- value
- data quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The application discloses a method and a device for detecting data quality abnormity, electronic equipment and a storage medium. According to the embodiment of the application, the sensor signal detected by the target sensor is obtained, the designated mathematical characteristics in the sensor signal are extracted, the designated mathematical characteristics are subjected to statistical analysis, whether the sensor signal has the drift characteristics or not is judged, and then the sensor signal is determined to have the data quality abnormity under the condition that the sensor signal has the drift characteristics, so that the technical problem that the drift characteristics cannot be identified by a detection method for the data quality abnormity in the related technology can be solved.
Description
Technical Field
The present application belongs to the field of data processing technologies, and in particular, to a method and an apparatus for detecting data quality abnormality, an electronic device, and a storage medium.
Background
The displacement sensor and the pressure sensor can be used for detecting displacement, deformation, pressure, tensile force and the like of mechanical equipment. For example, such sensors may be used to detect large component problems of wind turbine generators such as foundation cracking, main bearing play, flange clearance anomalies, and the like. However, the inventors found that when problems such as a mechanical mounting failure, a fixing adhesive failure, or a crack in a detection base, an abnormal flange gap, etc. occur, a drift characteristic that macroscopically rises or falls occurs in a signal of such a sensor, and in the related art, a detection method for an abnormal data quality cannot recognize the drift characteristic.
Disclosure of Invention
The embodiment of the application provides a method and a device for detecting data quality abnormity, electronic equipment and a storage medium, and can solve the technical problem that the drift characteristic cannot be identified by the method for detecting data quality abnormity in the related technology.
In a first aspect, an embodiment of the present application provides a method for detecting data quality abnormality, where the method includes:
acquiring a sensor signal detected by a target sensor, wherein the target sensor comprises a displacement sensor or a pressure sensor;
extracting a specified mathematical feature in the sensor signal;
carrying out statistical analysis on the specified mathematical characteristics, and judging whether the sensor signal has drift characteristics;
and determining that the sensor signal has data quality abnormity in the case that the sensor signal has the drift characteristics.
Optionally, acquiring a sensor signal detected by a displacement sensor or a pressure sensor includes:
acquiring an original signal acquired by a target sensor;
determining a signal analysis method corresponding to a signal communication mode of the target sensor;
and performing signal analysis on the original signal based on a signal analysis method to obtain a sensor signal.
Optionally, extracting the specified mathematical feature in the sensor signal comprises:
dividing the sensor signal into a plurality of periods of time with a specified duration;
calculating a value of a specified mathematical characteristic of the sensor signal for each time period, wherein the specified mathematical characteristic includes at least one of: average value, maximum value, minimum value and N quantile, wherein N is a positive integer greater than 1.
Optionally, statistically analyzing the specified mathematical features to determine whether there is a drift feature in the sensor signal, including:
determining that the sensor signal has a drift characteristic in the event that the values of the specified mathematical characteristic for the plurality of time periods satisfy the following statistical condition:
the absolute value of the difference between the value of the last time period designated mathematical feature and the value of the first time period designated mathematical feature exceeds a first threshold;
the cumulative sum of the differences between the value of the specified mathematical feature in each time period and the value of the specified mathematical feature in the previous time period exceeds the second threshold or does not exceed the third threshold.
Optionally, the statistical condition further comprises:
the number of times each time interval specifies that the difference between the value of the mathematical feature and the value of the mathematical feature specified in the previous time interval has the same sign exceeds the fourth threshold.
Optionally, before extracting the specified mathematical feature in the sensor signal, the method further comprises:
performing low pass filtering on the sensor signal; and/or the presence of a gas in the atmosphere,
removing data abnormal values in the sensor signal, wherein the data abnormal values comprise one of the following: singular values, signal frozen segments, communication outliers.
Optionally, in the case that the sensor signal has a drift characteristic, after determining that the sensor signal has a data quality abnormality, the method further includes:
and responding to the abnormality, and outputting an early warning prompt, wherein the early warning prompt is used for indicating that the target sensor has the abnormality.
On the other hand, an embodiment of the present application provides a device for detecting data quality abnormality, where the device includes:
the system comprises an acquisition unit, a processing unit and a control unit, wherein the acquisition unit is used for acquiring a sensor signal detected by a target sensor, and the target sensor comprises a displacement sensor or a pressure sensor;
an extraction unit for extracting a specified mathematical feature in the sensor signal;
the analysis unit is used for carrying out statistical analysis on the specified mathematical characteristics and judging whether the sensor signals have drift characteristics or not;
and the determining unit is used for determining that the sensor signal has data quality abnormity under the condition that the sensor signal has the drift characteristics.
Optionally, the obtaining unit includes:
the acquisition subunit is used for acquiring an original signal acquired by the target sensor;
a first determining subunit, configured to determine a signal analysis method corresponding to a signal communication manner of the target sensor;
and the analysis subunit is used for carrying out signal analysis on the original signal based on a signal analysis method to obtain a sensor signal.
Optionally, the extraction unit comprises:
a dividing subunit configured to divide the sensor signal into a plurality of periods at a specified duration;
a calculating subunit, configured to calculate a value of a specified mathematical characteristic of the sensor signal in each time interval, wherein the specified mathematical characteristic includes at least one of: average, maximum, minimum, N quantile, where N is a positive integer greater than 8.
Optionally, the analysis unit comprises:
a second determining subunit, configured to determine that the sensor signal has a drift characteristic if the value of the specified mathematical characteristic for the plurality of time periods satisfies the following statistical condition:
the absolute value of the difference between the value of the last time period designated mathematical feature and the value of the first time period designated mathematical feature exceeds a first threshold;
the cumulative sum of the differences between the value of the specified mathematical feature in each time period and the value of the specified mathematical feature in the previous time period exceeds the second threshold or does not exceed the third threshold.
Optionally, the statistical conditions further include:
the number of times each time period specifies that the numerical value of the mathematical feature is the same sign as the difference between the numerical values of the mathematical features specified in the previous time period exceeds the fourth threshold.
Optionally, the apparatus further comprises:
a filtering unit for performing low-pass filtering on the sensor signal before extracting a specified mathematical feature in the sensor signal; and/or the presence of a gas in the gas,
a data processing unit for eliminating data abnormal values in the sensor signal before extracting the specified mathematical features in the sensor signal, wherein the data abnormal values comprise one of the following: singular values, signal frozen segments, communication outliers.
Optionally, the apparatus further comprises:
and the output unit is used for responding to the data quality abnormity of the sensor signal after the sensor signal is determined to have the data abnormity under the condition that the sensor signal has the drift characteristic, and outputting an early warning prompt which is used for indicating that the target sensor has the abnormity.
In another aspect, an embodiment of the present application provides an electronic device, where the electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements a method of detecting data quality anomalies as in the first aspect.
In another aspect, an embodiment of the present application provides a storage medium, where the storage medium stores computer program instructions, and the computer program instructions, when executed by a processor, implement the method for detecting data quality abnormality according to the first aspect.
According to the data quality abnormity detection method, the device, the equipment and the storage medium, the specified mathematical characteristics in the sensor signals are extracted by obtaining the sensor signals detected by the target sensor, the specified mathematical characteristics are subjected to statistical analysis, whether the sensor signals have the drift characteristics or not is judged, and then the sensor signals are determined to have the data quality abnormity under the condition that the sensor signals have the drift characteristics, the drift characteristics existing in the signals of the displacement sensor or the pressure sensor can be identified, and the technical problem that the drift characteristics cannot be identified by a data quality abnormity detection method in the related technology can be solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly described below, and for those skilled in the art, other drawings may be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram illustrating a method for detecting data quality anomalies according to an embodiment of the present application;
FIG. 2 is a schematic diagram of data presence drift characteristics provided by one embodiment of the present application;
FIG. 3 is a schematic diagram of a data presence drift feature provided by another embodiment of the present application;
FIG. 4 is a schematic structural diagram of a method for detecting data quality abnormality according to another embodiment of the present application;
FIG. 5 is a schematic structural diagram of an apparatus for detecting data quality abnormality according to another embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to still another embodiment of the present application.
Detailed Description
Features and exemplary embodiments of various aspects of the present application will be described in detail below, and in order to make objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are intended to be illustrative only and are not intended to be limiting. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by illustrating examples thereof.
It should be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of another like element in a process, method, article, or apparatus that comprises the element.
In order to solve the problem of the prior art, embodiments of the present application provide a method, an apparatus, a device, and a storage medium for detecting data quality abnormality. First, a method for detecting data quality abnormality provided in the embodiment of the present application is described below.
In the data quality abnormality detection method provided by the embodiment of the application, the execution subject may be a data quality abnormality detection device, or a control module in the data quality abnormality detection device for executing the data quality abnormality detection method. In the embodiment of the present application, a method for detecting data quality abnormality by using a data quality abnormality detection device is taken as an example, and the data quality abnormality detection device provided in the embodiment of the present application is described.
For example, the execution main body of the data quality abnormality detection method may be a main controller of the fan, or a server running in a cloud, a server running in an electric field where the fan is located, or the like, and the execution main body may communicate with the target sensor in a wired or wireless communication manner, so as to acquire a required signal and output a detection result to a connected display terminal or other terminals in remote communication.
Fig. 1 is a flowchart illustrating a method for detecting data quality abnormality according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
The target sensor includes a displacement sensor or a pressure sensor. Displacement sensors and pressure sensors are typically sensors that can measure pressure, tension, deformation, etc. In an application scene, the target sensor can be used for detecting the problems of large parts of the wind generating set, such as base cracking, main bearing play, flange clearance abnormity and the like. Problems such as faulty mechanical installation, failure of the fixing adhesive and the like can cause the signal measured by the target sensor to have an abnormal characteristic of slow drift, namely a drift characteristic in the embodiment of the present application, exemplary drift characteristics are shown in fig. 2 and 3, fig. 2 is a schematic diagram of upward drift of the signal, and fig. 3 is a schematic diagram of downward drift of the signal.
Optionally, the step 101 of acquiring the sensor signal detected by the displacement sensor or the pressure sensor may include the steps of:
step 1011, acquiring an original signal acquired by a target sensor;
the raw signal refers to a signal directly acquired by the target sensor.
Step 1012, determining a signal analysis method corresponding to the signal communication mode of the target sensor;
and 1013, performing signal analysis on the original signal based on a signal analysis method to obtain a sensor signal.
In an application scenario, an executing party of the data quality anomaly detection method provided in the embodiment of the present application may not be able to identify the original signal acquired by the target sensor, and in this case, the original signal acquired by the target sensor needs to be analyzed. The specific signal analysis method is selected and determined through a signal communication mode with the target sensor, and the original signal is analyzed based on the determined signal analysis method, so that the sensor signal can be obtained. For example, the original signal transmitted by the MODBUS signal communication mode and the RS232/RS485 signal communication mode may be subjected to signal analysis based on the corresponding MODBUS signal analysis method and the RS232/RS485 signal analysis method, so as to obtain the sensor signal.
In step 102, a specified mathematical feature in the sensor signal is extracted.
The mathematical features may be statistical mathematical features, for example, the specified mathematical features may include at least one of: average, maximum, minimum, N-quantile, where N is a positive integer greater than 1, e.g., quartile, seventy-five quantile, etc.
For example, the average value Y of the continuous displacement signal Y (T) (i.e., the sensor signal) over a specified time period T mean Maximum value Y max Minimum value Y min Is defined as follows:
Y max ＝max(Y i ) Equation 2
Y min ＝min(Y i ) Equation 3
Optionally, low pass filtering may also be performed on the sensor signal before performing step 102 to extract the specified mathematical features in the sensor signal. Because the signal drift is a slow process, in order to block and weaken the influence of the signal fluctuation, first-order low-pass filtering or second-order and above low-pass filtering can be carried out on the displacement signal.
In one example, the sensor signal may be first divided into a plurality of cycles with a specified duration, so as to perform low-pass filtering according to equation 4, where C is a filter coefficient, and a typical value of C may be 5E (-5).
Y(t i )＝X(t i )*C+Y(t i-1 ) (1-C) formula 4
Here, X (t) i ) Representing the sensor signal, Y (t) i-1 ) Represents the value of the last cycle after filtering, Y (t) i ) Representing the filtered value of this period.
Optionally, before performing step 102 to extract the specified mathematical features in the sensor signal, in order to avoid the influence of individual cycle outliers, values of such outliers and cycles in the vicinity thereof may be removed in the filtering process, and specifically, removing data outliers in the sensor signal may include one of the following: singular values, signal frozen fragments, communication outliers.
And 103, performing statistical analysis on the specified mathematical characteristics, and judging whether the sensor signals have drifting characteristics.
Optionally, when step 102 is executed to extract the specified mathematical features in the sensor signal, the mathematical features may be extracted in segments, and specifically, the following steps may be included:
step 1021, the sensor signal is divided into a plurality of time segments with a specified duration.
An example value for the specified duration may be 1 day. For example, the sensor signal is divided into a plurality of periods (a plurality of cycles) with 1 day as a specified time length, and the signal in each period is processed separately.
At step 1022, a numerical value of the specified mathematical characteristic of the sensor signal for each time period is calculated.
The specified mathematical characteristics may include at least one of the specified mathematical characteristics of mean, maximum, minimum, N-quantile, etc., as described above.
Further, when performing step 103 to perform statistical analysis on the specified mathematical characteristics to determine whether the sensor signal has the drift characteristics, the sensor signal may be determined to have the drift characteristics if the values of the specified mathematical characteristics for a plurality of time periods satisfy the following statistical conditions:
statistical condition 1: the absolute value of the difference between the value of the last time period specific mathematical feature and the value of the first time period specific mathematical feature exceeds a first threshold.
For example, the maximum value, the minimum value and the average value in 20 consecutive specified time lengths T' can be calculated to form a maximum value array-max, a minimum value array-min and an average value array-mean. The array is arranged according to a time sequence, the first element is a specified mathematical characteristic of the first time period, and the last element is a specified mathematical characteristic of the latest time period.
And aiming at the maximum array-max, the minimum array-min and the mean array-mean, if the absolute value difference of the last element minus the first element is greater than a threshold value a, determining that the statistical condition 1 is met. A typical value for a may be 6 for the maximum array-max, the minimum array-min, and 4 for the mean array-mean, a.
Statistical condition 2: each time period specifies that the cumulative sum of the differences between the value of the mathematical feature and the value of the mathematical feature specified in the previous time period either exceeds the second threshold or does not exceed the third threshold.
And for the maximum value array-max, the minimum value array-min and the mean value array-mean, if the accumulated sum of the differences of the next element and the previous element in the adjacent elements is greater than a threshold b, determining that the statistical condition 2 is met. Typical values for b may be 7 for the maximum array-max, the minimum array-min, and 4 for the mean array-mean.
And if the judgment result of any one array of the maximum array-max, the minimum array-min and the mean array-mean simultaneously meets the statistical conditions 1 and 2, determining that the sensor signal has the drift characteristic.
Optionally, the statistical condition may further include:
statistical condition 3: the number of times each time period specifies that the numerical value of the mathematical feature is the same sign as the difference between the numerical values of the mathematical features specified in the previous time period exceeds the fourth threshold.
The sign of the difference between the numerical value of the specified mathematical characteristic in each time interval and the numerical value of the specified mathematical characteristic in the previous time interval represents the rising or falling of the trend of the data as a whole, if the rising or falling is continuous, the drift characteristic exists, and if the sign of the difference between the numerical value of the specified mathematical characteristic in each time interval and the numerical value of the specified mathematical characteristic in the previous time interval is the same, the data as a whole is drifted along the basically same trend.
And step 104, determining that the data quality of the sensor signal is abnormal under the condition that the sensor signal has the drift characteristics.
Optionally, after determining that the sensor signal has the data quality abnormality, an early warning prompt may be output in response to the abnormality, where the early warning prompt is used to indicate that the target sensor has the abnormality. In addition, a prompt of predictive operation and maintenance can be output.
According to the method, the device, the equipment and the storage medium for detecting the data quality abnormity, the specified mathematical characteristics in the sensor signals are extracted by acquiring the sensor signals detected by the target sensor, the specified mathematical characteristics are subjected to statistical analysis, whether the sensor signals have the drift characteristics or not is judged, and then the sensor signals are determined to have the data quality abnormity under the condition that the sensor signals have the drift characteristics, the drift characteristics existing in the signals of the displacement sensor or the pressure sensor can be identified, and the technical problem that the drift characteristics cannot be identified by a detection method for the data quality abnormity in the related technology can be solved.
According to the data quality abnormity detection method, the detectable drift signal can be acquired by a displacement sensor for detecting the main shaft movement, or can be acquired by a displacement sensor for detecting basic cracking, flange gap abnormity and the like. In addition, other types of sensors may be used, such as pressure sensors that measure pressure, tension, strain, and the like. The identification principle of the signal drift characteristics is the same, so that the method for detecting the data quality abnormality can be applied.
Fig. 5 is a block diagram of a structure of a method for detecting data quality abnormality according to another embodiment of the present application, which is mainly divided into three parts:
the first part is signal acquisition, and displacement signals can be acquired through a displacement sensor. Alternatively, the pressure sensor may collect a pressure signal, an elastic force signal, or the like.
The second part is analysis and judgment, specifically, the characteristics can be processed firstly, time intervals are divided for signals, designated mathematical characteristics in each time interval are counted, and whether the sensor signals have drift characteristics or not is judged according to whether the designated mathematical characteristics meet the statistical conditions of the drift characteristics or not.
The third part is early warning output, and under the condition that the drift characteristics of the sensor signals are determined, early warning prompts are output, and predictive operation and maintenance suggestions can be output.
Fig. 6 is a schematic structural diagram of an apparatus for detecting data quality abnormality according to an embodiment of the present application, and as shown in fig. 6, the apparatus includes an obtaining unit 201, an extracting unit 202, an analyzing unit 203, and a determining unit 204.
An acquisition unit 201, configured to acquire a sensor signal detected by a target sensor, where the target sensor includes a displacement sensor or a pressure sensor;
an extraction unit 202 for extracting a specified mathematical feature in the sensor signal;
the analysis unit 203 is used for performing statistical analysis on the specified mathematical characteristics and judging whether the sensor signals have drift characteristics;
a determining unit 204, configured to determine that the sensor signal has a data quality anomaly if the sensor signal has a drift characteristic.
Alternatively, the obtaining unit 201 may include:
the acquisition subunit is used for acquiring an original signal acquired by the target sensor;
a first determining subunit, configured to determine a signal analysis method corresponding to a signal communication manner of the target sensor;
and the analysis subunit is used for carrying out signal analysis on the original signal based on a signal analysis method to obtain a sensor signal.
Alternatively, the extraction unit 202 may include:
a dividing subunit for dividing the sensor signal into a plurality of periods with a specified duration;
a calculation subunit for calculating a value of a specified mathematical characteristic of the sensor signal in each time period, wherein the specified mathematical characteristic comprises at least one of: average, maximum, minimum, N quantile, where N is a positive integer greater than 8.
Alternatively, the analysis unit 203 may include:
a second determining subunit, configured to determine that the sensor signal has a drift characteristic if the value of the specified mathematical characteristic for a plurality of time periods satisfies the following statistical condition:
the absolute value of the difference between the value of the last time period designated mathematical feature and the value of the first time period designated mathematical feature exceeds a first threshold;
each time period specifies that the cumulative sum of the differences between the value of the mathematical feature and the value of the mathematical feature specified in the previous time period either exceeds the second threshold or does not exceed the third threshold.
Optionally, the statistical condition may further include:
the number of times each time interval specifies that the difference between the value of the mathematical feature and the value of the mathematical feature specified in the previous time interval has the same sign exceeds the fourth threshold.
Optionally, the apparatus may further include:
a filtering unit for performing low-pass filtering on the sensor signal before extracting a specified mathematical feature in the sensor signal; and/or the presence of a gas in the gas,
a data processing unit for eliminating data abnormal values in the sensor signal before extracting the specified mathematical features in the sensor signal, wherein the data abnormal values comprise one of the following: singular values, signal frozen fragments, communication outliers.
Optionally, the apparatus may further include:
and the output unit is used for responding to the data quality abnormity of the sensor signal after the sensor signal is determined to have the data quality abnormity under the condition that the sensor signal has the drift characteristic, and outputting an early warning prompt which is used for indicating that the target sensor has the abnormity.
According to the data quality abnormity detection method, the device, the equipment and the storage medium, the specified mathematical characteristics in the sensor signals are extracted by obtaining the sensor signals detected by the target sensor, the specified mathematical characteristics are subjected to statistical analysis, whether the sensor signals have the drift characteristics or not is judged, and then the sensor signals are determined to have the data quality abnormity under the condition that the sensor signals have the drift characteristics, the drift characteristics existing in the signals of the displacement sensor or the pressure sensor can be identified, and the technical problem that the drift characteristics cannot be identified by a data quality abnormity detection method in the related technology can be solved.
Fig. 6 shows a hardware structure diagram of an electronic device provided in an embodiment of the present application.
The electronic device may comprise a processor 301 and a memory 302 in which computer program instructions are stored.
Specifically, the processor 301 may include a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.
The memory may include Read Only Memory (ROM), random Access Memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical/tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software comprising computer-executable instructions and when the software is executed (e.g., by one or more processors), it is operable to perform operations described with reference to the methods according to an aspect of the application.
The processor 301 implements any one of the above-described data quality anomaly detection methods in the above-described embodiments by reading and executing computer program instructions stored in the memory 302.
In one example, the electronic device may also include a communication interface 303 and a bus 310. As shown in fig. 6, the processor 301, the memory 302, and the communication interface 303 are connected via a bus 310 to complete communication therebetween.
The communication interface 303 is mainly used for implementing communication between modules, apparatuses, units and/or devices in the embodiment of the present application.
It is to be understood that the present application is not limited to the particular arrangements and instrumentality described above and shown in the attached drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions or change the order between the steps after comprehending the spirit of the present application.
The functional blocks shown in the above-described structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be performed in an order different from the order in the embodiments, or may be performed simultaneously.
Aspects of the present application are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implement the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based computer instructions which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
As described above, only the specific embodiments of the present application are provided, and it can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the module and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. It should be understood that the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present application, and these modifications or substitutions should be covered within the scope of the present application.
Claims (16)
1. A method for detecting data quality abnormality is characterized by comprising the following steps:
acquiring a sensor signal detected by a target sensor, wherein the target sensor comprises a displacement sensor or a pressure sensor;
extracting a specified mathematical feature in the sensor signal;
performing statistical analysis on the specified mathematical characteristics, and judging whether the sensor signals have drift characteristics;
determining that the sensor signal has a data quality anomaly if the sensor signal has the drift characteristic.
2. The method for detecting data quality abnormality according to claim 1, wherein said acquiring a sensor signal detected by a displacement sensor or a pressure sensor includes:
acquiring an original signal acquired by the target sensor;
determining a signal analysis method corresponding to a signal communication mode of the target sensor;
and performing signal analysis on the original signal based on the signal analysis method to obtain the sensor signal.
3. The method of detecting data quality anomalies according to claim 1, characterized in that said extracting specified mathematical features in the sensor signal comprises:
dividing the sensor signal into a plurality of periods of time for a specified duration;
calculating a numerical value of the specified mathematical characteristic of the sensor signal for each time period, wherein the specified mathematical characteristic includes at least one of: average value, maximum value, minimum value and N quantile, wherein N is a positive integer greater than 1.
4. The method according to claim 3, wherein said statistically analyzing said specified mathematical features to determine if there is a drift feature in said sensor signal comprises:
determining that the drift characteristic is present in the sensor signal if the numerical value of the specified mathematical characteristic for the plurality of time periods satisfies the following statistical condition:
the absolute value of the difference between the value of the specified mathematical feature in the last time period and the value of the specified mathematical feature in the first time period exceeds a first threshold;
the accumulated sum of the differences between the value of the specified mathematical feature in each time period and the value of the specified mathematical feature in the previous time period exceeds a second threshold or does not exceed a third threshold.
5. The method according to claim 4, wherein the statistical conditions further include:
the number of times each time period the value of the specified mathematical feature differs from the value of the specified mathematical feature in the previous time period by the same sign exceeds a fourth threshold.
6. The method of detecting data quality anomalies according to claim 1, characterized in that, before said extracting specified mathematical features in the sensor signal, the method further comprises:
performing low pass filtering on the sensor signal; and/or the presence of a gas in the gas,
removing data outliers in the sensor signal, wherein the data outliers include one of: singular values, signal frozen fragments, communication outliers.
7. The method for detecting data quality abnormality according to claim 1, further comprising, after determining that there is a data quality abnormality in the sensor signal in a case where the drift characteristic exists in the sensor signal:
and responding to the abnormality, and outputting an early warning prompt which is used for indicating that the target sensor has the abnormality.
8. An apparatus for detecting data quality abnormality, comprising:
the system comprises an acquisition unit, a processing unit and a control unit, wherein the acquisition unit is used for acquiring a sensor signal detected by a target sensor, and the target sensor comprises a displacement sensor or a pressure sensor;
an extraction unit for extracting a specified mathematical feature in the sensor signal;
the analysis unit is used for carrying out statistical analysis on the specified mathematical characteristics and judging whether the sensor signals have drift characteristics or not;
a determining unit, configured to determine that there is a data quality abnormality in the sensor signal if the sensor signal has the drift characteristic.
9. The apparatus according to claim 8, wherein the acquiring unit includes:
the acquisition subunit is used for acquiring the original signal acquired by the target sensor;
a first determining subunit configured to determine a signal analysis method corresponding to a signal communication method of the target sensor;
and the analysis subunit is used for carrying out signal analysis on the original signal based on the signal analysis method to obtain the sensor signal.
10. The apparatus according to claim 8, wherein the extraction unit includes:
a dividing subunit configured to divide the sensor signal into a plurality of periods of time with a specified duration;
a calculating subunit for calculating a value of the specified mathematical characteristic of the sensor signal in each time period, wherein the specified mathematical characteristic comprises at least one of: average, maximum, minimum, N quantile, where N is a positive integer greater than 8.
11. The apparatus according to claim 10, wherein the analysis unit includes:
a second determining subunit, configured to determine that the drift characteristic exists in the sensor signal if the numerical value of the specified mathematical characteristic for the plurality of time periods satisfies the following statistical condition:
the absolute value of the difference between the value of the specified mathematical feature in the last time period and the value of the specified mathematical feature in the first time period exceeds a first threshold;
the accumulated sum of the differences between the value of the specified mathematical feature in each time period and the value of the specified mathematical feature in the previous time period exceeds a second threshold or does not exceed a third threshold.
12. The apparatus according to claim 11, wherein the statistical condition further comprises:
the number of times each time period the value of the specified mathematical feature differs from the value of the specified mathematical feature in the previous time period by the same sign exceeds a fourth threshold.
13. The apparatus for detecting data quality abnormality according to claim 8, characterized in that said apparatus further comprises:
a filtering unit for performing low pass filtering on the sensor signal before the extracting of the specified mathematical feature in the sensor signal; and/or the presence of a gas in the gas,
a data processing unit for eliminating data outliers in the sensor signal prior to the extracting of the specified mathematical features in the sensor signal, wherein the data outliers include one of: singular values, signal frozen fragments, communication outliers.
14. The apparatus for detecting data quality abnormality according to claim 8, characterized in that said apparatus further comprises:
and the output unit is used for responding to the existence of the abnormity after determining that the data quality of the sensor signal is abnormal under the condition that the sensor signal has the drift characteristic, and outputting an early warning prompt which is used for indicating that the target sensor has the abnormity.
15. An electronic device, characterized in that the electronic device comprises: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements a method of detecting data quality anomalies as claimed in any one of claims 1 to 7.
16. A storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method of detecting data quality anomalies as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111153105.7A CN115905341A (en) | 2021-09-29 | 2021-09-29 | Data quality abnormity detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111153105.7A CN115905341A (en) | 2021-09-29 | 2021-09-29 | Data quality abnormity detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115905341A true CN115905341A (en) | 2023-04-04 |
Family
ID=86495210
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111153105.7A Pending CN115905341A (en) | 2021-09-29 | 2021-09-29 | Data quality abnormity detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115905341A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117249873A (en) * | 2023-11-20 | 2023-12-19 | 精智未来(广州)智能科技有限公司 | Quality monitoring method and equipment for gas molecular analysis |
| CN117292843A (en) * | 2023-11-24 | 2023-12-26 | 苏州百孝医疗科技有限公司 | Electrical signal data processing method, apparatus, device and storage medium |
-
2021
- 2021-09-29 CN CN202111153105.7A patent/CN115905341A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117249873A (en) * | 2023-11-20 | 2023-12-19 | 精智未来(广州)智能科技有限公司 | Quality monitoring method and equipment for gas molecular analysis |
| CN117249873B (en) * | 2023-11-20 | 2024-01-30 | 精智未来(广州)智能科技有限公司 | Quality monitoring method and equipment for gas molecular analysis |
| CN117292843A (en) * | 2023-11-24 | 2023-12-26 | 苏州百孝医疗科技有限公司 | Electrical signal data processing method, apparatus, device and storage medium |
| CN117292843B (en) * | 2023-11-24 | 2024-02-06 | 苏州百孝医疗科技有限公司 | Electrical signal data processing method, apparatus, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115905341A (en) | Data quality abnormity detection method and device, electronic equipment and storage medium | |
| EP3399376B1 (en) | Plant-abnormality-monitoring method and computer program for plant abnormality monitoring | |
| CN113918376B (en) | Fault detection method, device, equipment and computer readable storage medium | |
| CN113284143B (en) | Die casting deburring precision detection system based on image data processing | |
| CN110906508B (en) | Fault detection method and system for air conditioner sensor | |
| CN110678820A (en) | Abnormality importance calculation system, abnormality importance calculation device, and abnormality importance calculation program | |
| CN117407679B (en) | Data acquisition method and system of intelligent end screen sensor | |
| CN116308305B (en) | Bridge health monitoring data management system | |
| CN114844762B (en) | Alarm authenticity detection method and device | |
| CN114993640A (en) | Equipment state monitoring method, device, equipment and computer storage medium | |
| CN116685967A (en) | Method and device for detecting anomalies in a machine or machine component | |
| CN112945531A (en) | Method, device and equipment for detecting cracks of fan blade and computer storage medium | |
| JP5751606B2 (en) | Abnormality diagnosis system for machinery | |
| CN114576102A (en) | Method, device and equipment for predicting bearing state of wind turbine generator and storage medium | |
| JP5354174B2 (en) | Abnormality diagnosis system for machinery | |
| CN109870404B (en) | Rain shed structure damage identification method and device and terminal equipment | |
| CN110307899A (en) | Sound anomaly detection system based on deep learning | |
| CN112814890A (en) | Method for detecting pump machine fault based on voiceprint and vibration | |
| CN115905907A (en) | Abnormity monitoring method and device for fan main bearing, electronic equipment and storage medium | |
| CN113614253A (en) | Blast furnace abnormality determination device, blast furnace abnormality determination method, and blast furnace operation method | |
| CN117433591B (en) | Automatic equipment monitoring system based on multi-sensor data | |
| CN117191305B (en) | State evaluation method and system for highway bridge | |
| CN117803633A (en) | Oil leakage detection method, device and equipment of hydraulic system and computer storage medium | |
| CN117556108B (en) | Abnormal detection method for oil-residue separation efficiency based on data analysis | |
| JP4354496B2 (en) | Traffic leap detection device and traffic leap detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |