KR102489502B1 - Sensor outlier detection system for building monitoring using ensemble algorithm - Google Patents

Abstract

In operating of a vibration, crack, tilt sensor, and the like used for safety monitoring of a building, the present invention relates to a sensor outlier detection system for building monitoring using an ensemble algorithm that sequentially applies a plurality of detection algorithms, enables to effectively detect an outlier of a detection signal generated by various causes, and enables to improve a precision of monitoring by responding to an error. The sensor outlier detection system for building monitoring using the ensemble algorithm comprises: a first detection module; a second detection module; a third detection module; a fourth detection module; and a verification module.

Description

Sensor outlier detection system for building monitoring using ensemble algorithm {Sensor outlier detection system for building monitoring using ensemble algorithm}

The present invention relates to a sensor anomaly detection system, and more particularly, in the operation of vibration, crack, and tilt sensors used for safety monitoring of buildings, a plurality of detection algorithms are sequentially applied to detect anomalies in detection signals caused by various causes. It relates to a sensor anomaly detection system for building monitoring using an ensemble algorithm that can improve the precision of monitoring by effectively detecting values and responding to errors.

Outliers, which occur in various sensors and are defined as data that are not within the observable range or that are very unlikely to be observed, or data that have different observed attribute values from other data, are called outliers, extreme values, It is called a singular value, etc., and may have an unexpected effect on the stability of an established model in modeling data, or may indicate an anomaly or a unique characteristic point in collected data, preventing trend changes or occurrence of important issues. It is a very important value in terms of detection.

Vibration, crack, and tilt sensors used in the building safety monitoring system are input in the form of time-series data due to the nature of IoT sensor data, and are affected by temperature, rain, built-in power battery exhaustion, and sensor outages due to building construction. However, there are many elements that can be considered as outliers that are heterogeneous in the collected data due to actual sharp cracks or tilts in the building. Therefore, it is necessary to detect outliers and take appropriate measures in the preprocessing stage prior to data modeling for the safety monitoring system and in the visualization of data.

In addition, unlike other fields where errors in outlier detection are not a relatively big problem, accurate detection of outliers in safety-related data is the biggest reason for the existence of the entire system due to the nature of building safety, so a strict outlier detection model is required.

Since the causes of these outliers are diverse, finding out whether data are outliers often involves the researcher's subjectivity or expertise by reflecting various factors. In other words, since outliers occur due to various reasons, such as relatively simple reasons outside the normal range where outliers can occur, measurement errors, communication errors, recording errors, and values arising from various reasons that deviate from the usual operating environment, it is necessary to detect outliers. This should be done taking into account the context of data collection and the impact it will have on the model.

Sensors that measure vibration, cracks, and tilts used in building monitoring also deviate from the trend and have a large variance for a short period of time (additive outlier), or show a different trend for a relatively long period of time and then return to the original level (transient Outliers appear in various forms, such as a change outlier), a case that changes significantly and continues to change (level shift), and a case where a large change continues to affect subsequent observed values (innovational outlier).

However, as the outlier detection technique has various causes, it is difficult to draw a clear conclusion that which one is good or bad because the algorithm used is vast and the logical basis of the technique used is diverse, and the results are also different.

The outlier search of general data currently being implemented is divided into univariate and multivariate cases, and whether each observation is a normal value or an outlier, builds an outlier detection model with prior information, and assumes that the data have a specific probability distribution Depending on whether it can be seen as following, various outlier detection techniques can be used. In addition, there are methods to detect outliers while fitting an ARIMA model by taking advantage of the characteristics of time series data, and methods to search for outliers using nonparametric quantile regression.

A serious problem in detecting such outliers is actually the “poverty in abundance” of the methodology. There are various algorithms depending on the type of data. standards do not exist

Generally, the GOF (goodness of fit) test method is usually used in statistical methodology, which is based on the difference between the measured value and the predicted value. In outlier search, this method is used unless it is a supervised method that knows in advance whether a specific data is an outlier. Hard to do.

Republic of Korea Patent No. 10-2274389 (2021.07.01)

The present invention was created to solve the above problems, and an object of the present invention is to sequentially apply a plurality of algorithms to abnormal values of detection signals generated from various causes in monitoring the safety of buildings using various sensors. To provide a sensor outlier detection system for building monitoring using an ensemble algorithm that detects and responds to errors to obtain accurate detection results.

For the above object, the present invention provides an anomaly detection system for a sensor installed in a building to monitor safety, an arranging unit for arranging data values obtained from the sensor in ascending order, and dividing the sorted data values at a set ratio. a first detection module having a section setting unit configured to set a boundary section by doing so, and defining and storing a data value outside the boundary section as a first outlier; a second detection module that normalizes data values obtained from the sensor, tests them at a set confidence level with reference to a standard normal distribution, and extracts and stores second outliers; a test statistic unit configured to set an upper limit on the number of the first and second outliers and to calculate a test statistic; and a threshold setter configured to set a threshold value to be compared with the test statistic, wherein the test statistic is larger than the set threshold value. a third detection module for defining and storing the case as a third outlier; a fourth detection module configured to set a score calculation unit that calculates an outlier score using an isolation forest algorithm, and to set a ratio of outliers and to define and store an outlier score data value that meets the set ratio as a fourth outlier; Based on the first outlier, the second outlier, the third outlier, and the fourth outlier, an entropy calculation unit that calculates entropy for a dataset from which original data values and outlier data values have been removed, and the change in entropy from the perspective of probability fluctuation a verification module having an analysis unit that reinterprets as , and comparing a set value with a probability change to determine whether an outlier is extracted; It is characterized by consisting of.

At this time, the sensor may be a sensor capable of measuring vibration, crack, and tilt.

In addition, the interval setting unit divides the data values organized in ascending order into 4 equal parts, multiplies the difference between the 75% percentile value and the 25% value by 1.5, and sets the maximum value added to the percentile 75% value and the minimum value subtracted from the percentile 25% value. It is possible to set a boundary section to

In addition, the second detection module may test whether there is an outlier at a 95 to 99% confidence level.

In addition, the test statistics unit sets the upper limit of the number of outliers by adding 1 to the maximum values of the first and second outliers, calculates the sample mean and sample standard deviation, and calculates the maximum value of the deviation between the observed value and the sample mean. , remove the data value corresponding to the maximum value from the sample, and calculate the test statistic again to calculate the test statistic up to the number of outliers of the estimated maximum value.

Various existing methods for outlier detection have difficulties in application due to various limitations such as the need to estimate the number of outliers in advance, the assumption that the data must follow a normal distribution, and the assumption that the data must be balanced on both sides at the median. In addition, the results are different depending on the application of different methodologies, so the outlier detection results depend on which methodology is used. In addition, since there is no measurement tool that objectively explains how appropriate the outlier detection result has become, it is difficult to take follow-up measures according to the detection result.

The present invention overcomes the problems of various previously applied methods and applies an ensemble technique that simultaneously mixes and applies several techniques so that accurate outlier detection can be achieved, so that the collected data meets the assumptions required by the technique and prevents unfulfillment. Since it is possible to dispel concerns about the following, apply methodologies sequentially to satisfy mutually necessary assumptions, and check whether or not an outlier has been detected appropriately based on the amount of entropy information for outliers extracted by various methodologies, stable outlier detection can be expected. It works.

In this way, the method of using statistics on the amount of information can be applied not only to univariate techniques but also to multivariate outlier search results, so the outlier detection algorithms, which are currently crowded with dozens of other algorithms, can be mutually evaluated on the basis of the change in the amount of information. can provide a basis for

1 is a conceptual diagram of the present invention;
2 is a block diagram showing the configuration and connection relationship according to an embodiment of the present invention;
3 is a flow chart showing a sequence according to an embodiment of the present invention;
4 to 5 are sheets showing examples of detecting anomalies according to experimental examples of the present invention;
6 to 9 are sheets showing analysis results for sensor data values according to experimental examples of the present invention.

Hereinafter, the configuration of a sensor anomaly detection system for monitoring a building using the ensemble algorithm of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram of the present invention. The present invention is a system for detecting anomalies in a sensor installed in a building to monitor safety. The detection algorithm is based on the following four algorithms. This has the advantage of being able to detect outliers without relying on a specific methodology, because even if other techniques are added, the outliers can be cleaned up by following the same steps.

First, as IQR (Interquatile range), data are arranged in order of size, and the interval between the 1st quartile (25%) and 3rd quartile (75%) is called IQR. If the data value is larger or smaller than the value obtained by multiplying this IQR value by 1.5 and adding or subtracting the median value, it is considered an outlier.

Although there is no distributional assumption for special data and no subjective factors are included in the interpretation of the results, for unbalanced data (when the frequency of both data around the median is very different), outlier search is different from other methods and still Because outliers are detected based on distance, there is a problem in detecting outliers when two disparate data groups are mixed (mixed distribution).

Second, it is an algorithm called z-score or standardized score, which shows how far the data are from the mean value by considering the standard deviation, under the assumption that the data follow a normal probability distribution, and is expressed as follows.

,

는 자료값,

는 평균,

는 표준편차의 제곱근이다.

,

is the data value,

is the mean,

is the square root of the standard deviation.

The z-score causes a problem with widely used algorithms or assumptions that the data follow a normal distribution, or because the average reacts sensitively to the extreme values if a lot of extreme values are mixed in the data. Whether to use a 95% confidence coefficient or a 99% confidence coefficient here belongs to the subjective judgment area of the user. However, if the normal distribution assumption is satisfied, it shows the most sensitive response to outlier detection and has the advantage of being intuitive.

Thirdly, as a generalized ESD test, it is tested using the same statistics as the Grubb test, but it is a method that supplements the weakness of the grubb test. The weakness of having to determine the number of outliers in advance is alleviated and only the upper limit of the number of outliers needs to be specified.

Although it is based on the distance of the outlier, it has the theoretical advantage that the probability of a type 1 error in outlier judgment for several points is mathematically corrected and calculated. The ESD test only needs to present an upper limit on the number of outliers, and has the advantage of simultaneously detecting multiple outliers. However, it is still based on normality and shares the weakness of distance-based outlier detection, and in large data, the simulation results for the numerical type 1 error probability of the test become unstable.

For reference, the Grubb test (Grubb test) verifies whether it is an outlier under the assumption that the data come from a normal distribution with the absolute deviation of the data that is the farthest from the average value among the data values.

This test is performed on one of the data with the largest deviation, and the test is repeated until the outlier is no longer detected. As the number of samples decreases, the probability of concluding that it is an outlier when it is not actually an outlier increases, so the number of samples increases. It is necessary to stop the black when it gets smaller.

Next, as the fourth isolation forest, this technique continues to generate straight lines that divide data values into two at random with a straight line until all data are divided into straight lines, and then relatively quickly distinguish data from other data as outliers. It is a technique to

The i-Forest technique, which is a kind of simulation technique that scores and judges the average value by applying the same algorithm several times, has recently come out with an extended version with a straight line with a slope. ) is a methodology specialized for outlier detection.

Outlier detection calculation time is relatively fast and it is easy to use because it does not make any special assumptions, but it is not easy to interpret the result because you have to input in advance how many data out of the total data to be considered as outliers.

This Isolation Forest is one of Unsupervised Anomaly Detection and is mainly used to detect outliers among existing data. In particular, it has the advantage of being able to operate efficiently even in data with many variables.

The concept of the Isolation Forest is that isolating (= isolating) each observation is that outliers are easier than normal data. is close to and the path length is small.

Just as random forests ensemble decision trees by iterating multiple times, Isolation Forest ensembles iTree iterations multiple times. iTree is ① Sub-sampling: Sampling some of the data with non-repair extraction, ② Variable selection: Randomly selecting q among the variables of data X, ③ Setting a split point: Split point uniformly among the range of variable q (max~min) Select, ④ Processes ① to ③ are repeated until all observations are split or a random number of splits (=recursive tree), and all path lengths are saved. will be repeated several times.

Evaluation is done as follows.

where h ( x ): path length of that observation, E ( h ( x )): average path length for that observation across all iTree, c ( n ): value to normalize h ( x ), average of iTree path length. (since iTree has the same structure as Binary Search Tree, c ( n ) is easily obtained)

The score value according to E ( h ( x )) is 1. Observation value x is similar to the average of all path lengths (= normal data): E ( h ( x )) → c ( n ), s → 0.5, 2. Observation value x is an outlier: E ( h ( x ))→0, s →1, 3. Maximum path length of observation x: E ( h ( x ))→ n-1, s →0, that is, Score is 0 to It is distributed between 1, and the closer to 1, the more likely it is to be an outlier, and if it is less than 0.5, it can be judged as normal data.

In the present invention, a so-called ensemble algorithm that can accurately detect outliers and counteract errors by organically combining these algorithms is used.

Looking at the problem of the existing outlier detection methodology, as shown in the example using actual data in the following content, the basis theory and the use situation are different, so when applied to actual data, there are cases where the result of outlier detection is similar, but the result is different. There are many different cases.

In particular, in the case of a building safety diagnosis platform, since the detection of an outlier is related to the meaning of the existence of the platform itself, apart from the problem of prediction, it is not a desirable phenomenon for results to vary greatly depending on the methodology used to require accuracy, and Even the criterion for judging whether something is suitable for the situation is an ambiguous reality.

Because this method uses different grounds to develop a unique argument for selecting heterogeneous data, methodologies using different grounds cannot theoretically distinguish superiority and inferiority. This is because it is rare to be able to do post-hoc verification.

Therefore, in the present invention, the suitability of detecting outliers using entropy information was determined.

In data science, entropy means the amount of information, and Shannon entropy or the average amount of information is used as entropy for the amount of information.

The amount of information is defined as the log of the reciprocal of the probability that an event will occur, and the average of this amount of information is called the entropy value. Entropy values are small for highly probable events and provide large values for unlikely events.

From the point of view of outliers, it is reasonable to select the outlier detection value as the methodology that gives the smallest entropy value by comparing the entropy of data excluding outliers detected through various methods. In other words, excluding outliers (values with a small probability of occurrence), the average amount of information provided by the rest of the data excluding the outliers must be significantly greater than the average amount of information calculated from the data including the outliers, and each different Outliers can be used for a suitability test integrating various methodologies that can determine which outlier extraction method is more suitable in terms of information amount by comparing the average amount of information of a data set excluding the extracted outliers.

(엔트로피)의 정의는 다음과 같다.average amount of information

The definition of (entropy) is as follows.

Here, I(X) is defined as the amount of information of individual data, and b is the base of the logarithmic function. In Shannon entropy, 2, the number of bits, is used, and the natural number 10 or Euler's constant e is sometimes used.

As a result of detecting outliers with various outlier extraction methodologies, it is better to select cases in which the average amount of information in the data set excluding outliers is significantly smaller. Entropy has a characteristic that a smaller value appears when data with a low probability of occurrence are excluded from the data.

However, even if it is not an outlier with a very small probability, the value becomes small even if data with a relatively small probability value are excluded. When data with a very high probability is excluded, the entropy value increases, and since it is calculated by taking the logarithm of the probability value, it has a non-linear character with respect to the probability value.

Since the difference between the data excluding outliers and the original data is difficult to intuitively interpret, it is easy to feel the difference in information more intuitively by converting the average information into the change in the average probability of the data.

Therefore, in the present invention, the difference in average information amount is defined as follows. If H(X) is used as the definition of the average amount of information, theoretical calculation becomes difficult because the probability distribution also changes when outliers are excluded.

Therefore, the average amount of information is regarded as the representative amount of information I(X) of the dataset, and the entropy of the average amount of information is regarded as the representative amount of information of the full data set and the data excluding outliers (reduced data set). The difference in the amount of representative information of the two data is,

으로 표현될 수 있다.

can be expressed as

Considering the average amount of information in the two data sets as the representative amount of information and examining the difference in terms of probability, we can say that the greater the difference in probability, the better the outlier extraction.

If there is not much difference in the probability value, we can say that the outlier extraction result is poor.

As various algorithms for detecting outliers in conventional sensors are provided, there are advantages and disadvantages according to the methodology, and it is also a problem that various algorithms can be used only when the number of outliers is specified in advance.

Among various algorithms, outlier detection methods that fit a time-series model and use differences from predicted values are state-of-the-art and have proven effective, but were excluded because special model fitting is not necessarily required due to the nature of building-related data.

In addition, since the multivariate method is not required and the building-related sensor data is not a situation in which data on stability are accumulated, only unsupervised methods can be used. Due to the nature of sensor data as time series data, outlier detection was performed with first-order residuals to exclude the effect of time.

2 is a block diagram showing configuration and connection relationships according to an embodiment of the present invention, and FIG. 3 is a flowchart showing a sequence according to an embodiment of the present invention. In the present invention, four representative univariate outlier detection methodologies are simultaneously applied, We propose an ensemble algorithm that calculates the entropy of the result detected as an outlier according to each methodology and defines it as an outlier in the sensor dataset based on the change in the probability value of the representative information.

To this end, the tilt sensor is first converted from the Cartesian coordinate system to the polar coordinate system as an example. At this time, the value corresponding to the angle is not considered, and only the magnitude of the slope is considered in the outlier detection. The magnitude of the gradient value is expressed as:

First, in the first step (S 110), an outlier is detected by the IQR (Inter quatile range) method, which includes the sorting unit 111 that sorts the data values obtained from the sensor in ascending order, and the sorted data values at a set ratio. It is performed through a first detection module 110 that includes a section setting unit 112 that divides and sets a boundary section, and defines and stores a data value outside the boundary section as a first outlier.

That is, the data values are read in ascending order and divided into 4 equal parts to calculate the difference between the 75% percentile and the 25% percentile. This value is called IQR. At this time, the interval setting unit 112 multiplies IQR by 1.5 to set a boundary interval in which the maximum value added to the 75% percentile value and the minimum value obtained by subtracting the 25% percentile value are minimized. Points outside this range are defined as outliers, and the values detected as outliers and the number of outliers are stored.

In the next second step (S 120), outliers are detected by the Z-Score method, which normalizes the data values obtained from the sensor, and tests at the set confidence level with reference to the standard normal distribution to extract the second outliers and It is solved through the second detection module 120 to store.

That is, assuming that the data values are normally distributed, the mean and standard deviation are calculated to calculate the score. In fact, cracks and tilt sensors related to building safety take the form of a normal distribution of white noise under normal circumstances, and when abnormalities occur, unusual trends or outliers appear, so it can be considered a reasonable assumption.

At this time, the second detection module 120 checks whether or not the calculated score is an outlier at a 95% or 99% confidence level by referring to a standard normal distribution. Stores calculated outliers and results.

In the next third step (S 130), an outlier is detected using the generalized ESD test, a test statistics unit 131 sets an upper limit for the number of first outliers and second outliers and calculates a test statistic, and the test statistic It is performed through a third detection module 130 that includes a threshold value setting unit 132 for setting a threshold value to be compared with the threshold value, and defines and stores the test statistic as a third outlier when the test statistic is greater than the set threshold value.

That is, the test statistics unit 131 sets the upper limit of the number of outliers to the maximum value of the first step (S 110) and the second step (S 120) plus 1, and then calculates the test statistic, but the sample mean and sample standard deviation are calculated, the maximum value of the deviation between the observed value and the sample mean is calculated, the data value corresponding to the maximum value is removed from the sample, and the test statistic is calculated again to calculate the test statistic up to the number of outliers of the estimated maximum value. This is expressed as:

.

은 추정된 이상치의 최대값.

.

is the maximum value of the estimated outlier.

In addition, the threshold value setting unit 132 calculates a threshold value to be compared with the test statistic as follows.

분포의 100p 백분율 값을 의미하고

로 정의한다. 계산된

가 임계값

보다 크면 이상치로 간주하고 이 값을 저장한다. 이 검정은

인 경우 상당히 정확하다.where t(p,v) is t with degrees of freedom v

means the 100p percentage value of the distribution, and

is defined as calculated

is the threshold

If it is greater than , it is regarded as an outlier and this value is stored. this test

If , it is fairly accurate.

In the next fourth step (S 140), an outlier is detected using the isolation forest technique, which is achieved by using the isolation forest algorithm to calculate the outlier score, the score calculation unit 151, setting the ratio of outliers, and It is performed through the fourth detection module 140 that defines and stores the outlier score data value as the fourth outlier.

This is a technique for detecting outliers using a decision tree. After preemptively estimating how much outliers will occupy in the data, a spatial segmentation algorithm is used to detect outliers that have no similarities with other observations.

Many outlier detection techniques use statistical distances between data, but have the disadvantage of exponentially increasing the amount of calculation when there are many data or multidimensional data. It is a little more free from the problem of classifying an outlier as a normal value when there is a group of outliers and swamping (when a normal value is mistakenly classified as an outlier due to its proximity to the outlier).

그 외의 자료는

로 나눈다.

혹은

에 데이터가 2개 이상 포함되 있다면 이것을 후보 리스트 집합인 C에 다시 포함 시키고 그렇지 않다면 이것을 고립된 노드로 간주한다. 모든 자료들이 고립된 노드로 구분될 때까지 같은 작업을 반복하고 자료별로 모두 고립될 때까지 필요한 작업의 수를 저장하고 필요 작업수가 작은 것들을 이상치로 간주한다.The algorithm builds a list (C) containing the root node with full data and candidate nodes for further splitting. One node (R) is randomly extracted from this list and this node is deleted from list C. For the selected R, a characteristic value (a) is randomly set, and data less than or equal to a

other data

divide by

or

If two or more data are included in , it is included again in C, the candidate list set, and if not, it is regarded as an isolated node. The same operation is repeated until all data are classified as isolated nodes, and the number of required operations is stored for each data until all of them are isolated, and those with a small number of required operations are considered outliers.

This algorithm calculates an anomaly score as a result, and the one with a high outlier score is declared an outlier. In the present invention, outlier detection is performed by reflecting the maximum value of the number of outliers suggested in other methodologies. However, iForest sets the ratio of outliers in advance and declares the top % of outlier scores that fit the ratio as outliers. runs as far as computing outlier scores.

In the fifth and final step (S 150), entropy is calculated for the data set from which the original data and the data detected as outliers are removed based on the detected outlier data after sequentially applying the four outlier detection techniques.

Based on the first outlier, the second outlier, the third outlier, and the fourth outlier, the entropy calculation unit 151 calculates the entropy of the data set from which the original data value and the outlier data value have been removed, and the change in entropy. It is provided with an analysis unit 152 that reinterprets in terms of probability fluctuation, and is performed through a verification module 150 that determines the quality of outlier extraction by comparing the set value with the probability fluctuation.

Entropy calculations require proper discretization of the data. Create a discrete probability frequency distribution table by dividing the normal range from which data are derived into an appropriate number, and identify the frequencies that fall between each class.

Based on this data, entropy is calculated, and the change in entropy, which is the average amount of information, is reinterpreted in terms of the probability change of data to interpret the change in entropy due to outliers. If the fluctuation of this probability is large, it is concluded that outlier detection is successful, and if not, there is a problem with the outlier extraction detected by the methodology.

4 and 5 are sheets showing examples of outlier detection according to experimental examples of the present invention, and it can be seen that the detection of outliers emitted by different methodologies coincides and the change in the amount of information is large.

As with other statistical hypothesis tests, the subjectivity of the user or data interpreter intervenes in how large the change in the amount of information must be to be considered a significant change in the amount of information in terms of the standard of change in the representative probability value. A solution may exist if the probability distribution of the statistic can be accurately derived, but a mathematical answer is difficult unless the assumption of the probability distribution of the statistic is made.

For example, assuming that the data values follow a normal distribution, the information content function can be derived as follows.

Assuming a normal distribution, the average entropy increases as the variance increases and decreases as the variance decreases. However, if the assumption of probability distribution of data values is made, there is a problem in dealing with methodologies that do not assume normality among various methodologies.

In order to circumvent this, in the present invention, when the average entropy value is converted to a probability value standard and observed, a probability change of about 1% per one outlier data is assumed to be a significant change.

Conversion from the point of view of probability values is to view the average amount of information as a representative value of the amount of information in data and convert the value of information amount back into probability for observation. After applying it to the actual data, the interpreter can estimate the approximate amount of significant information change based on the outlier detection data based on the converted probability values.

6 to 9 are sheets showing analysis results for sensor data values according to experimental examples of the present invention.

Figure 6 shows the results in which outliers were not detected through different methods, and in Figure 7, the same outliers were detected in IQR and z-score, but the generalized ESD test showed no outliers detected, and the fluctuations in probability values varied between IQR and z-score. Since the outlier data detected in -score exceeded 5%, the two extracted outliers were accepted as outliers.

In FIG. 8, one was detected only in the z-score methodology, but the probability change did not exceed 1%, so it was concluded that there was no outlier. Including the results, 4 were detected as outliers, and when 4 were viewed as outliers, the probability value change exceeded 15%, and it was concluded that 4 were regarded as outliers.

In this way, by comparing the results of different outlier detection algorithms at the same time, more accuracy was given to the specific detected outlier, and the probability change value was obtained as a basis, and confusion and distrust due to the detection of different results for each individual methodology could be eliminated.

The rights of the present invention are defined by what is described in the claims, not limited to the embodiments described above, and that those skilled in the art can make various modifications and adaptations within the scope of rights described in the claims. It is self-evident.

110: first detection module 111: alignment unit
112: section setting unit 120: second detection module
130: third detection module 131: verification statistics unit
132: threshold value setting unit 140: fourth detection module
141: score calculation unit 150: verification module
151: entropy calculation unit 152: analysis unit

Claims (5)

In the system for detecting anomalies of sensors installed in buildings to monitor safety,
An arranging unit 111 for sorting the data values obtained from the sensor in ascending order, and a section setting unit 112 for setting a boundary section by dividing the sorted data values at a set ratio, the data values outside the boundary section a first detection module 110 defining and storing as a first outlier;
a second detection module (120) for normalizing the data values obtained from the sensor and for extracting and storing second outliers by testing them at a set confidence level with reference to the standard normal distribution;
The upper limit of the number of the first and second outliers is set and the test statistic is calculated, but the upper limit of the number of outliers is set as the value obtained by adding 1 to the maximum value of the first and second outliers, and the sample mean and sample standard deviation are calculated. A test statistics unit 131 that calculates the maximum value of the deviation between the observed value and the sample mean, removes the data value corresponding to the maximum value from the sample, calculates the test statistic again, and calculates the test statistic up to the number of outliers of the estimated maximum value; a third detection module 130 having a threshold setting unit 132 for setting a threshold to be compared with the test statistic, and defining and storing the test statistic as a third outlier when the test statistic is greater than the set threshold;
a score calculator 151 that calculates an outlier score using an isolation forest algorithm, and a fourth detection module 140 that sets a ratio of outliers and defines and stores an outlier score data value that meets the set ratio as a fourth outlier;
Based on the first outlier, the second outlier, the third outlier, and the fourth outlier, the entropy calculation unit 151 calculates entropy for the dataset from which original data values and outlier data values have been removed, and the change in entropy a verification module 150 having an analysis unit 152 that reinterprets from the point of view of probability fluctuation, and comparing a set value with a probability change to determine whether an outlier is extracted; Sensor anomaly detection system for building monitoring using an ensemble algorithm, characterized in that consisting of.
According to claim 1,
The sensor is a sensor anomaly detection system for monitoring buildings using an ensemble algorithm, characterized in that the sensor is a sensor capable of measuring vibration, cracks, and tilt.
According to claim 1,
The section setting unit 112,
The data values organized in ascending order are divided into 4 parts, and the difference between the 75% and 25% values is multiplied by 1.5 to set a boundary section that maximizes the value added to the 75% value and minimizes the value subtracted from the 25% value. A sensor anomaly detection system for building monitoring using an ensemble algorithm.
According to claim 1,
The second detection module 120,
A sensor anomaly detection system for monitoring a building using an ensemble algorithm, characterized in that it tests for outliers at a 95 to 99% confidence level.
delete

Images