CN106934208B - A kind of dam thundering observed data automatic identifying method - Google Patents
A kind of dam thundering observed data automatic identifying method Download PDFInfo
- Publication number
- CN106934208B CN106934208B CN201710006186.5A CN201710006186A CN106934208B CN 106934208 B CN106934208 B CN 106934208B CN 201710006186 A CN201710006186 A CN 201710006186A CN 106934208 B CN106934208 B CN 106934208B
- Authority
- CN
- China
- Prior art keywords
- sequence
- dam
- feature group
- matrix
- identifying method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Z—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS, NOT OTHERWISE PROVIDED FOR
- G16Z99/00—Subject matter not provided for in other main groups of this subclass
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Complex Calculations (AREA)
Abstract
The present invention discloses a kind of dam thundering observed data automatic identifying method, comprising: the track matrix of building primary monitoring data sequence carries out singular value decomposition to track matrix and obtains feature group；Feature group is arranged from big to small according to characteristic value, selecting several the preceding feature groups of accumulation contribution rate more than or equal to 85% is main feature group；The corresponding fundamental matrix of main feature group is calculated, diagonal equalization is carried out to fundamental matrix and obtains the main component of data sequence；Main component is added up and obtains reconstruct data sequence；Subtract each other to obtain residual sequence with reproducing sequence and former data sequence, finds out the standard deviation of residual sequence；Judge whether measured value is exceptional value by Pauta criterion according to the residual error of measured value.This method can automatically extract the main feature of monitoring data sequence, avoid artificial founding mathematical models, not only can guarantee the consistency and accuracy of judgement, also reduce human resources investment；It still is able to differentiate monitoring data when the environment parameters such as water level, temperature missing.
Description
Technical field
The present invention relates to Dam Monitoring field more particularly to a kind of dam thundering observed data automatic identifying methods.
Background technique
China builds together into 100 meters or more more than 200 seats of dam, wherein 150 meters or more of more than 40 seat of high dam.To these dams into
Row monitoring, monitoring are to guarantee the important means of its safe operation." the Dam operational safety prison that Bureau of Energy in 2015 promulgates
Superintend and direct administrative provisions " " for 100 meters of height of dam or more of dam, electric power enterprise should establish dam safety and supervise online further requirement
Control system ", " dam center should carry out remote online technical supervision to the dam operational safety that registration (putting on record) registers ".Due to
Operating condition is complicated, and the monitoring point that high dam is laid is usually far more than common engineering；And when being managed to group dam, the survey that is related to
Point number is even more huge.
The vital task of on-line monitoring is identification abnormal operating condition, and basis is identification thundering observed data.Currently,
The visible outlier identification method in Dam Monitoring field is divided into method of identification based on envelope domain according to its principle, based on item
The method of identification of part similitude and method of identification three classes based on mathematical model.Wherein, the exception based on statistical regression mathematical model
Value recognition methods can be normally reached higher susceptibility, while also be not easy for normal value to be mistaken for exception, therefore application is most
It is extensive.But the people that the foundation of regression model is typically required for abundant data analysis experience operates completion.When to monitoring point pole
When more high dams or group dam is monitored, human cost is huge；Meanwhile different operators be likely to occur standard it is inconsistent,
The even situation of inaccuracy.In addition, also typically including the environment parameters such as water level, temperature in statistical model, and pass through in practical projects
Often there is the case where reporting and submitting when environment parameter and effect quantity difference, at this moment will be unable to obtain effect quantity predicted value using statistical model,
Thus also can not just judge whether measured value is abnormal.
Summary of the invention
Manual operation and environment parameter are relied in the dam thundering observed data identification technology based on statistical model
The excessively high problem of degree, the present invention provide a kind of method of automatic identification thundering observed data, it is intended to promote dam safety monitor
Precision, efficiency and the robustness that abnormal value identifies in work, and reduce human resources investment.
To achieve the goals above, the present invention discloses a kind of dam thundering observed data automatic identifying method, the method
The following steps are included:
1) by primary monitoring data sequence construct track matrix, singular value decomposition then is carried out to track matrix and obtains a system
Column feature group；
2) feature group is arranged from big to small according to characteristic value, selection accumulation contribution rate more than or equal to 85% it is preceding several
Feature group is main feature group；
3) the corresponding fundamental matrix of main feature group is calculated, diagonal equalization then is carried out to fundamental matrix and obtains data sequence
Several preceding main components of column；
4) main component obtained is added up and obtains reconstruct data sequence；
5) subtract each other to obtain residual sequence with reproducing sequence and former data sequence, find out the standard deviation of residual sequence；
6) judge whether measured value is exceptional value by Pauta criterion according to the standard deviation of residual sequence.
Further, in above-mentioned steps disclosed by the invention, the track matrix X is by monitoring data sequence f0,f1,
f2,...,fN-1Lag arrangement in time obtains, and indicates are as follows:
Wherein, N is monitoring data sequent data count, and L is length of window, and 1 < L < N, K includes by the track every row of matrix X
Measured value quantity, K=N-L+1；I, j are used to indicate element xijIt is to be arranged in the i-th row jth the location of in the matrix X of track.
Further, described that track matrix X progress singular value decomposition is referred to: to seek S=XXTNon-negative eigenvalue λ1,λ2,
λ3,...,λlAnd the feature vector U of corresponding orthonormalization1,U2,U3,...,UlAndThe spy
Sign group refers to (λi,Ui,Vi), referred to as ith feature group.
Further, the contribution rate CR of the ith feature groupiIt is calculated by following formula:
Further, the main feature group is the preceding m feature group accumulated contribution rate and be more than or equal to 85%, it may be assumed that
It is that i, j are used to indicate for which characteristic value, and m indicates that the sum of main feature group, l indicate the total of non-negative feature group
Number.
Further, the fundamental matrix XiIt is calculated by feature group:
Further, described to fundamental matrix XiThe process diagonally equalized are as follows: for the matrix X of L × Ki, enable it
Element is xij, L*=min (L, K), K*=max (L, K), N=L+K-1；If L < K, xij *=xij, otherwise xij *=xji.It is described
I-th of main componentIt is calculated as follows:
The reconstruct data sequence is obtained by preceding m main component is cumulative, it may be assumed that
The residual sequence ΔkSubtract each other to obtain by reproducing sequence and former data sequence, it may be assumed that
When the Pauta criterion is that take confidence level be 99%, the standard deviation of residual sequence is σ, when following formula is set up
Measured value is normal value, is otherwise exceptional value:
|Δk|≤2.58σ (14)
The invention has the benefit that monitoring data can be automatically extracted using the outlier identification method that the present invention announces
The main feature of sequence, avoids artificial founding mathematical models, not only can guarantee the consistency and accuracy of judgement, also substantially reduces
Human resources investments.Meanwhile it still being able to differentiate monitoring data when the environment parameters such as water level, temperature missing.
Detailed description of the invention
Fig. 1 is the flow chart of one embodiment of method of the invention.
Fig. 2 is measured data to be analyzed in one embodiment, is a top of arch dam displacement monitoring data.
Fig. 3 is the contribution rate distribution map of preceding 20 feature groups in one embodiment.
Fig. 4 is the corresponding main component curve graph of CR1=67.1% in one embodiment.
Fig. 5 is the corresponding main component curve graph of CR2=12% in one embodiment.
Fig. 6 is the corresponding main component curve graph of CR3=11.98% in one embodiment.
Fig. 7 is the reconstruct data sequence obtained in one embodiment by first three main component.
Fig. 8 is the residual sequence acquired in one embodiment by reconstruct data sequence and measured data sequence and reasonable residual error
Range.
Specific embodiment
Below in conjunction with attached drawing and one embodiment to the reality of Dam Monitoring Data outlier identification method of the present invention
It applies mode and beneficial effect is described further.Fig. 1 is the flow chart for the outlier identification method announced according to the present invention, is used
This method carries out analysis identification to top of arch dam displacement monitoring data shown in Fig. 2.The following steps are included:
Step 1: reading monitoring data sequence f0,f1,f2,...,fN-1, data sequence is lagged to arrangement in time and is obtained
Track matrix X:
Wherein, N is monitoring data sequent data count, and L is known as length of window, 1 < L < N；For there is periodic monitoring data
Sequence, it is desirable that the value of L is greater than the length of a cycle.L=400 days, N=6800 days in the present embodiment.
Step 2: singular value decomposition being carried out to track matrix X: seeking matrix S=XXT, seek the non-negative eigenvalue λ of S1,λ2,
λ3,...,λlAnd the feature vector U of corresponding orthonormalization1,U2,U3,...,UlAndObtain l
Feature group (λi,Ui,Vi), i=1,2 ... .l.
Step 3: seeking the contribution rate CR of ith feature groupi, its calculation formula is:
Wherein, the non-negative characteristic value sum that l is matrix S.
Step 4: feature group being arranged from big to small according to characteristic value, the accumulation contribution rate SCR of i feature group before askingi:
The accumulation contribution rate SCR of i feature group before judgingiWhether it is more than or equal to 85%, if set up, enables main feature
Group number m=i, and choosing this m feature group is main feature group；Otherwise, the cumulative tribute for seeking next feature group is continued through
Offer rate.
Fig. 3 is the contribution rate distribution map of preceding 20 feature groups of embodiment illustrated in fig. 2, it is seen that the contribution of first three feature group
Rate be noticeably greater than after feature group, accumulation contribution rate be 91%；Therefore, select first three feature group for the master of the present embodiment
Want feature group.
Step 5: calculating the corresponding fundamental matrix of main feature group: the corresponding fundamental matrix X of i-th of main feature groupiAre as follows:
Wherein, i=1,2 ..., m.
Step 6: diagonal equalization being carried out to the corresponding fundamental matrix of main feature group and seeks main component: for the square of L × K
Battle array Xi, enabling its element is xij, L*=min (L, K), K*=max (L, K), N=L+K-1；If L < K, xij *=xij, otherwise xij *
=xji.I-th of main componentIt is calculated as follows:
Fig. 4 show first three corresponding main component of main feature group of embodiment illustrated in fig. 2.
Step 7: preceding m main component being added up and obtains reconstruct data sequence, it may be assumed that
Wherein, k=0,1,2 ..., N-1.Heavy line show the reconstruct data sequence of corresponding embodiment illustrated in fig. 2 in Fig. 5
Column are obtained by first three corresponding main component of main feature group is cumulative.
Step 8: subtracting each other to obtain residual sequence Δ by reproducing sequence and former data sequencek, it may be assumed that
Wherein, k=0,1,2 ..., N-1.Fig. 6 show the residual sequence of corresponding embodiment illustrated in fig. 2.
Step 9: find out the standard deviation sigma of residual sequence:
The standard deviation of the residual error of embodiment illustrated in fig. 2 is 0.9090.
Step 10: for each measured value, judge whether measured value is exceptional value according to Pauta criterion:
|Δk|≤2.58σ (9)
When above formula is set up, measured value corresponding to residual error is normal value；Otherwise the probability that the residual error corresponds to measured value appearance is less than
1%, thus be exceptional value.
The grey area Fig. 6 is residual error range ± 2.3452 of normal measured value in the embodiment determined according to above-mentioned criterion,
Measured value corresponding to residual error except the region is abnormal value.
Claims (9)
1. a kind of dam thundering observed data automatic identifying method, which is characterized in that the described method comprises the following steps:
1) by primary monitoring data sequence construct track matrix, singular value decomposition then is carried out to track matrix and obtains a series of spies
Sign group；
2) feature group is arranged from big to small according to characteristic value, selection accumulation contribution rate is more than or equal to 85% several preceding features
Group is main feature group；
3) the corresponding fundamental matrix of main feature group is calculated, diagonal equalization then is carried out to fundamental matrix and obtains data sequence
Several preceding main components；
4) main component is added up and obtains reconstruct data sequence；
5) subtract each other to obtain residual sequence with reconstruct data sequence and former data sequence, find out the standard deviation of residual sequence；
6) judge whether measured value is exceptional value by Pauta criterion according to the standard deviation of residual sequence.
2. dam thundering observed data automatic identifying method according to claim 1, which is characterized in that the track matrix
X is by monitoring data sequence f0,f1,f2,...,fN-1Lag arrangement in time obtains, and indicates are as follows:
Wherein, N is monitoring data sequent data count, and L is length of window, the measured value that 1 < L < N, K includes by the track every row of matrix X
Quantity, K=N-L+1；I, j are used to indicate element xijIt is to be arranged in the i-th row jth the location of in the matrix X of track.
3. dam thundering observed data automatic identifying method according to claim 1, which is characterized in that described to track square
Battle array X carries out singular value decomposition and refers to: seeking S=XXTNon-negative eigenvalue λ1,λ2,λ3,...,λlAnd corresponding orthonormalization
Feature vector U1,U2,U3,...,UlAndThe feature group refers to (λi,Ui,Vi), referred to as ith feature
Group.
4. dam thundering observed data automatic identifying method according to claim 3, which is characterized in that described i-th special
The contribution rate CR of sign groupiIt is calculated by following formula:
The main feature group is the preceding m feature group accumulated contribution rate and be more than or equal to 85%, it may be assumed that
It is that i, j are used to indicate for which characteristic value, and m indicates that the sum of main feature group, l indicate the sum of non-negative feature group.
5. dam thundering observed data automatic identifying method according to claim 1, which is characterized in that the fundamental matrix
XiIt is calculated by following formula:
6. dam thundering observed data automatic identifying method according to claim 1, which is characterized in that described to basic square
Battle array XiCarry out the process that diagonal equalization seeks main component are as follows: for the matrix X of L × Ki, enabling its element is xij, L*=min (L,
K), K*=max (L, K), N=L+K-1；If L < K, xij*=xij, otherwise xij*=xji, i-th of main componentIt is calculated as follows:
7. dam thundering observed data automatic identifying method according to claim 1, which is characterized in that the reconstruct data
Sequence is obtained by preceding m main component is cumulative, it may be assumed that
8. dam thundering observed data automatic identifying method according to claim 1, which is characterized in that the residual sequence
ΔkSubtract each other to obtain by reproducing sequence and former data sequence, it may be assumed that
9. dam thundering observed data automatic identifying method according to claim 1, which is characterized in that the La Yida is quasi-
When being then that take confidence level be 99%, the standard deviation of residual sequence is σ, otherwise it is different that when following formula is set up, measured value, which is normal value,
Constant value:
|Δk|≤2.58σ (8)。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710006186.5A CN106934208B (en) | 2017-01-05 | 2017-01-05 | A kind of dam thundering observed data automatic identifying method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710006186.5A CN106934208B (en) | 2017-01-05 | 2017-01-05 | A kind of dam thundering observed data automatic identifying method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106934208A CN106934208A (en) | 2017-07-07 |
| CN106934208B true CN106934208B (en) | 2019-07-23 |
Family
ID=59444700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710006186.5A Active CN106934208B (en) | 2017-01-05 | 2017-01-05 | A kind of dam thundering observed data automatic identifying method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106934208B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111177218B (en) * | 2019-12-25 | 2022-08-30 | 深圳市东深电子股份有限公司 | Dam safety analysis method based on big data analysis |
| CN111080982A (en) * | 2019-12-31 | 2020-04-28 | 三峡大学 | Dam safety intelligent monitoring and early warning system and method based on multiple sensors |
| CN111680267B (en) * | 2020-06-01 | 2022-04-01 | 四川大学 | Three-step online identification method for abnormal dam safety monitoring data |
| CN112487356B (en) * | 2020-11-30 | 2022-10-14 | 北京航空航天大学 | Structural health monitoring data enhancement method |
| CN112541538B (en) * | 2020-12-11 | 2023-01-31 | 中国电建集团华东勘测设计研究院有限公司 | Dam abnormity monitoring data automatic identification method under poor information condition |
| CN112507591B (en) * | 2020-12-11 | 2022-11-18 | 四川大学 | Gravity dam crest displacement monitoring method based on separation type hybrid modeling technology |
| CN116128312B (en) * | 2023-04-17 | 2023-07-14 | 南昌工程学院 | Dam safety early warning method and system based on monitoring data analysis |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8380435B2 (en) * | 2010-05-06 | 2013-02-19 | Exxonmobil Upstream Research Company | Windowed statistical analysis for anomaly detection in geophysical datasets |
| CN103699874A (en) * | 2013-10-28 | 2014-04-02 | 中国计量学院 | Crowd abnormal behavior identification method based on SURF (Speed-Up Robust Feature) stream and LLE (Locally Linear Embedding) sparse representation |
| CN104181883A (en) * | 2014-08-08 | 2014-12-03 | 青岛高校信息产业有限公司 | Method for processing abnormal data of real-time data acquisition system in real time |
| CN104750861A (en) * | 2015-04-16 | 2015-07-01 | 中国电力科学研究院 | Method and system for cleaning mass data of energy storage power station |
| CN105205112A (en) * | 2015-09-01 | 2015-12-30 | 西安交通大学 | System and method for excavating abnormal features of time series data |
| CN105894014A (en) * | 2016-03-29 | 2016-08-24 | 中国人民解放军海军航空工程学院 | Abnormal behavior sequential detection method based on multi-factor inconsistency |
-
2017
- 2017-01-05 CN CN201710006186.5A patent/CN106934208B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8380435B2 (en) * | 2010-05-06 | 2013-02-19 | Exxonmobil Upstream Research Company | Windowed statistical analysis for anomaly detection in geophysical datasets |
| CN103699874A (en) * | 2013-10-28 | 2014-04-02 | 中国计量学院 | Crowd abnormal behavior identification method based on SURF (Speed-Up Robust Feature) stream and LLE (Locally Linear Embedding) sparse representation |
| CN104181883A (en) * | 2014-08-08 | 2014-12-03 | 青岛高校信息产业有限公司 | Method for processing abnormal data of real-time data acquisition system in real time |
| CN104750861A (en) * | 2015-04-16 | 2015-07-01 | 中国电力科学研究院 | Method and system for cleaning mass data of energy storage power station |
| CN105205112A (en) * | 2015-09-01 | 2015-12-30 | 西安交通大学 | System and method for excavating abnormal features of time series data |
| CN105894014A (en) * | 2016-03-29 | 2016-08-24 | 中国人民解放军海军航空工程学院 | Abnormal behavior sequential detection method based on multi-factor inconsistency |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106934208A (en) | 2017-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106934208B (en) | A kind of dam thundering observed data automatic identifying method | |
| Vazirizade et al. | Seismic reliability assessment of structures using artificial neural network | |
| CN116451879B (en) | Drought risk prediction method and system and electronic equipment | |
| CN108375715B (en) | Power distribution network line fault risk day prediction method and system | |
| CN110472846A (en) | Nuclear power plant's thermal-hydraulic safety analysis the best-estimated adds uncertain method | |
| CN109470946B (en) | Power generation equipment fault detection method and system | |
| CN104156615A (en) | Sensor test data point anomaly detection method based on LS-SVM | |
| Li et al. | Hydrostatic seasonal state model for monitoring data analysis of concrete dams | |
| CN104537211A (en) | Enterprise safety risk early warning method based on analytic hierarchy process and grey theory | |
| Hwang et al. | Long-term damping characteristics of twin cable-stayed bridge under environmental and operational variations | |
| CN108960329A (en) | A kind of chemical process fault detection method comprising missing data | |
| CN115507822A (en) | Flood risk prediction method under hydrographic cyclic variation drive | |
| CN105320987A (en) | Satellite telemetry data intelligent interpretation method based on BP neural network | |
| Burr et al. | Revisiting statistical aspects of nuclear material accounting | |
| Aliberti et al. | Indoor air-temperature forecast for energy-efficient management in smart buildings | |
| CN105046089B (en) | A kind of method for predicting heavy showers and flood | |
| CN112700050B (en) | Method and system for predicting ultra-short-term 1 st point power of photovoltaic power station | |
| CN109961160B (en) | Power grid future operation trend estimation method and system based on tide parameters | |
| CN103197564B (en) | Digitizing man-machine interface monitor unit quantity optimization method and system | |
| Hou et al. | An artificial neural network based anomaly detection algorithm for nuclear power plants | |
| Khoroshilov | Mathematical Modelling of Sayano-Shushenskaya Dam Displacement Process after 2009 Accident | |
| CN108599147B (en) | Combined interval prediction method based on normal exponential smoothing method and kernel density estimation | |
| CN109033564A (en) | A kind of aircraft airborne equipment testability demonstration index evaluating method | |
| Feng et al. | An integrated reliability model of hardware-software system | |
| CN112260211A (en) | Method and device for determining operating state of relay protection device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190528 Address after: No. 22 Chaowang Road, Hangzhou, Zhejiang Province, Zhejiang Applicant after: LARGE DAM SAFETY SUPERVISION CENTER, NATIONAL ENERGY ADMINISTRATION OF THE PEOPLE'S REPUBLIC OF CHINA Applicant after: Co., Ltd of East China Investigation and Design Institute of electricity Jian group of China Address before: No. 22 Chaowang Road, Hangzhou, Zhejiang Province, Zhejiang Applicant before: Co., Ltd of East China Investigation and Design Institute of electricity Jian group of China |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |