CN106649579A - Time-series data cleaning method for pipe net modeling - Google Patents
Time-series data cleaning method for pipe net modeling Download PDFInfo
- Publication number
- CN106649579A CN106649579A CN201611011961.8A CN201611011961A CN106649579A CN 106649579 A CN106649579 A CN 106649579A CN 201611011961 A CN201611011961 A CN 201611011961A CN 106649579 A CN106649579 A CN 106649579A
- Authority
- CN
- China
- Prior art keywords
- data
- value
- cleaning method
- pipe net
- series data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a time-series data cleaning method for pipe net modeling. The time-series data cleaning method for the pipe net modeling comprises the steps of searching and elimination of duplicate values, data dispersion degree analysis, judgment of outliers, denoising noisy points of curve smoothing, interpolation completion of missing data. The time-series data cleaning method for the pipe net modeling introduces variation coefficient to achieve standardization processing of pressure information and flow data of different dimensions, judges the dispersion degree of arrays and screen the dispersion degree of arrays at the same time. The time-series data cleaning method for the pipe net modeling is characterized in that outlier data at first is searched and processed by a utilizing three times standard deviation method and then is fitted by a least square method, which greatly reduces the effects of outliers on fitting results. At the same time, data smoothing of the noisy points is processed by fitting functions, which can further reduce the presence of outlier data. The least square method can satisfy the data processing which does not conform to the normal distribution. Compared with linear interpolation, cubic spline interpolation utilized in the end can make the data inserted more smooth. The time-series data cleaning method for the pipe net modeling has the advantages of preprocessing the data before the data is imported into a model for calculation, achieving the effect of data cleaning, and providing a guarantee for the calculation of the model.
Description
Technical field
The invention belongs to technical field of data processing, clear in particular to a kind of time series data for pipe net modeling
Washing method.
Background technology
It is related to a large amount of Monitoring Datas during pipe net modeling to process, what is be for example related to is main with seasonal effect in time series data
There are the Monitoring Data of water factory's discharge pressure and water flow, Income gap, water-use model data, for model checking
Pressure, data on flows of pipe network monitoring point etc..But, some are correct in these data, and some are then due to mechanical device
, can inevitably there are some time point exceptional value, shortage of data, Data duplications etc. and ask in some not specific factors such as error
Topic.If do not screened, certain impact will certainly be produced to the model calculation it could even be possible to directly resulting in model meter
Not do not restrain, the generation of the phenomenon such as model collapse, therefore we were needed into line number before these data to be imported to model calculating
According to pretreatment make up to the effect of cleaning, the calculating for model provides guarantee.
For example, in the sequential Monitoring Data collected, other abnormal numerical value sometimes occurs, intuitively,
This data is more small than other data or much larger.When test data is processed, for such indivedual exceptional values, it is
No to reject, how polishing after rejecting, if intuitively judged merely, lacks theoretic foundation.For modeling Monitoring Data
In above-mentioned exceptional value, shortage of data, Data duplication problem, there is presently no the standardization flow process of complete set.Generally exist
For exceptional value is only artificial judgment reasonable interval during modeling, for the process of missing values and exceptional value is to ignore missing values to use
Simple linear difference is supplementing exceptional value.
The content of the invention
It is an object of the invention to provide a kind of time series data cleaning method for pipe net modeling, the method is for pipe network
Time series data in modeling, the data prediction mode that can take relatively reasonable science provides guarantor for the precision that model is calculated
Card.
To realize above-mentioned technical purpose, above-mentioned technique effect is reached, the present invention is achieved through the following technical solutions：
A kind of time series data cleaning method for pipe net modeling, comprises the following steps：
Step 1）Repetition values are sifted out；
Using SQL（SQL）The data of section the time required to choosing, the data of same monitoring site are entered as one group
Row repetition values are searched, and delete the repetition values of same time point；
Step 2）Dispersion degree is analyzed；
Batch calculates respectively different group data maximums Xmax, minimum of a value Xmin, average value mu, standard deviation sigma and coefficient of variation CV, its
Middle CV=σ/μ, by standard deviation sigma and coefficient of variation CV come the dispersion degree of analyze data, can be by by the process of coefficient of variation CV
The same batch processed of flow and pressure data of different dimensions；And to coefficient of variation CV given threshold, when the coefficient of variation is more than institute
During the threshold value of setting, then the data of the monitoring site are judged as invalid data, and deleted, be not involved in model calculating；
Step 3）Exceptional value judges；
Upper lower limit value is determined by triple standard difference method, i.e. normal value X is, determine the upper limit
It is worth and is, determine that lower limit is, reject for the value for not meeting this scope is exceptional value；
Step 4）Smoothed curve goes noise；
For each group monitoring point for removing exceptional value（Discrete point）Data adopt least square fitting smoothed curve, first really
A fixed functionApproach original function；If approximate function is, functional valueWith observationDifference
Referred to as residual error, can weigh approximate function with residual errorQuality, concrete grammar is：
According to known data point, first with MATLAB solving equations, undetermined coefficient and fitting function are obtained；Recycle fitting function
Value replaces curve noise value, reaches the effect of curve smoothing；Further, can will replace fitting function value after noise value to enter again
Row fitting, repeat the above steps are until residual error meets required precision；
Step 5）Interpolation processing is carried out to missing values；
Row interpolation is entered to missing values using cubic spline function, the time series data repetition for processing monitoring is described by above-mentioned steps
The larger sequence data of value, missing values, exceptional value and dispersion；
When carrying out data processing in actual modeling process, the function that least square fitting goes out most to approach observation is first passed through,
The trend trend of general control data, while screening step 3）In fail by three times standard deviation send out remove exceptional value and pick
Remove, reduce the presence of error；
When model data is actually imported, the data of local segmentation are recycled, using spline interpolation method by missing values
And the part of abnormality value removing carries out polishing, to prevent the distortion of matched curve data, while remaining former rational observation.
Further, step 1）In, described time period data include the monitoring number of water factory's discharge pressure and water flow
According to, Income gap, water-use model data, and pressure, the flow of the different pipe network monitoring point positions for model checking
Time series data.
Further, step 2）In, the threshold value of the coefficient of variation may be set to 1, i.e. standard deviation sigma less than average value mu, real
In trampling as coefficient of variation ＜ 1, the pressure and flow time series data discrete degree monitored is preferable.
Further, in step 4）In, described functionCurve do not required on the graph it is all of
Data point（Error impact can be eliminated）, but need to show the trend of data as far as possible, near these data points.
The invention has the beneficial effects as follows：
The invention provides the judgement of exceptional value, the standardization of the pressure data data on flows of different dimensions, using difference
Significance analysis are to the quick lookup of exceptional value and the method for replacement, while selecting most reasonably to insert after being compared missing data
A whole set of the flow chart of data processing such as value mode.By introducing the coefficient of variation（Standard deviation/average）To realize different dimensions
Pressure data and data on flows standardization, can carry out judging the dispersion degree of array and screening simultaneously.The present invention is in side
First with triple standard difference method for least square fitting is used again in the process of exceptional value data search in method, exceptional value is significantly reduced
Impact to fitting result；Simultaneously with fitting function to the further presence for reducing abnormal data of noise data smoothing process,
Least square fitting disclosure satisfy that the data processing for not meeting normal distribution；Finally using cubic spline interpolation compared with linear interpolation
The numerical value that insertion can be made more is smoothed.Therefore the method for the present invention can import data to model enter it before calculating
Row pretreatment, to reach the effect of data cleansing, the calculating for model provides guarantee.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,
And can be practiced according to the content of specification, below with presently preferred embodiments of the present invention and coordinate accompanying drawing describe in detail as after.
The specific embodiment of the present invention is shown in detail in by following examples and its accompanying drawing.
Description of the drawings
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the application, this
Bright schematic description and description does not constitute inappropriate limitation of the present invention for explaining the present invention.In the accompanying drawings：
Fig. 1 is the flow chart of the time series data cleaning method of the present invention.
Specific embodiment
Below with reference to the accompanying drawings and in conjunction with the embodiments describing the present invention in detail.
With reference to shown in Fig. 1, a kind of time series data cleaning method for pipe net modeling is comprised the following steps：
Step 1）Repetition values are sifted out
Using SQL（SQL）The data of section the time required to choosing, described time period data include water factory's water outlet
The Monitoring Data of pressure and water flow, Income gap, water-use model data, and for the difference pipe of model checking
The pressure of net monitoring site, flow time series data；The data of same monitoring site carry out repetition values lookup as one group, and delete
Except the repetition values of same time point.
Step 2）Dispersion degree is analyzed
Batch calculates respectively different group data maximums Xmax, minimum of a value Xmin, average value mu, standard deviation sigma and coefficient of variation CV.
If this group of numerical value X1, X2, X3... XnIts mean value（Arithmetic mean of instantaneous value）For μ；Then standard deviation sigma is：
The coefficient of variation is：CV=σ/μ.
By standard deviation sigma and coefficient of variation CV come the dispersion degree of analyze data, can be by by the process of coefficient of variation CV
The same batch processed of flow and pressure data of different dimensions；And to coefficient of variation CV given threshold, when the coefficient of variation is more than institute
During the threshold value of setting, then the data of the monitoring site are judged as invalid data, and deleted, be not involved in model calculating.
The data that certain monitoring site part-time section collects generally are had in actual modeling and is 0, remaining time points
According to normal, with actual conditions and do not meet, this group of data are invalid data, the standard deviation and the coefficient of variation of such data compared with
Greatly, therefore removal can be analyzed by dispersion.
In practical experience as coefficient of variation ＜ 1, the data discrete degree monitored is preferable, for what is monitored in modeling
The threshold value of the coefficient of variation of flow and pressure time series data may be selected to be 1, i.e. standard deviation sigma less than average value mu.
Step 3）Exceptional value judges
Upper lower limit value is determined by triple standard difference method, i.e. normal value X is, determine the upper limit
It is worth and is, determine that lower limit is, reject for the value for not meeting this scope is exceptional value.For symbol
The value data for closing normal distribution is distributed in（μ -3 σ, μ+3 σ）In probability be 0.9974, therefore data outside the interval
It is regarded as exceptional value.
Step 4）Smoothed curve goes noise；
For each group monitoring point for removing exceptional value（Discrete point）Data adopt least square fitting smoothed curve, first really
A fixed functionOriginal function is approached, the curve of the function did not require on the graph all of data point（Can be with
Eliminating error affects）, but the function needs to show the trend of data as far as possible, near these data points.
If approximate function is, functional valueWith observationDifference be referred to as residual error, can be weighed with residual error
Amount approximate functionQuality, concrete methods of realizing is as follows：
If known data point, seek m order polynomials
Carry out fitting function.Need to obtain the polynomial undetermined coefficient of m+1 items, and following functional value is reached most
It is little：
；
Make above-mentioned function reach minimum of a value, had by higher mathematics knowledge：
；
I.e.
；
Then normal equation is obtained：
；
It is converted into matrix as follows
；
Using MATLAB solving equations, undetermined coefficient and fitting function are obtained.
Replace curve noise value using fitting function value, reach the effect of curve smoothing.Further noise value can be replaced
It is fitted again afterwards, repeat the above steps are until residual error meets required precision.
Step 5）Interpolation processing is carried out to missing values
Row interpolation is entered to missing values using cubic spline function, the time series data repetition for processing monitoring is described by above-mentioned steps
The larger sequence data of value, missing values, exceptional value and dispersion；
When carrying out data processing in actual modeling process, the function that least square fitting goes out most to approach observation is first passed through,
The trend trend of general control data, while screening step 3）In fail by three times standard deviation send out remove exceptional value and pick
Remove, reduce the presence of error；
When model data is actually imported, the data of local segmentation are recycled, using spline interpolation method by missing values
And the part of abnormality value removing carries out polishing, concrete methods of realizing is as follows：
In [a, b] superior functionCubic spline functions S (x) meet：
（1）0,1,2 mediation number is continuous on [a, b], i.e.,
；
（2）；
（3）In intervalOnIt is cubic polynomial.
Polishing is carried out to the exceptional value and missing values of rejecting by above-mentioned interpolation processing, to prevent the mistake of matched curve data
Very, while remaining former rational observation.
The preferred embodiments of the present invention are the foregoing is only, the present invention is not limited to, for those skilled in the art
For member, the present invention can have various modifications and variations.All any modifications within the spirit and principles in the present invention, made,
Equivalent, improvement etc., should be included within the scope of the present invention.
Claims (4)
1. a kind of time series data cleaning method for pipe net modeling, it is characterised in that comprise the following steps：
Step 1）Repetition values are sifted out；
The data of section the time required to being chosen using SQL, the data of same monitoring site carry out weight as one group
Complex value is searched, and deletes the repetition values of same time point；
Step 2）Dispersion degree is analyzed；
Batch calculates respectively different group data maximums Xmax, minimum of a value Xmin, average value mu, standard deviation sigma and coefficient of variation CV, its
Middle CV=σ/μ, by standard deviation sigma and coefficient of variation CV come the dispersion degree of analyze data, can be by by the process of coefficient of variation CV
The same batch processed of flow and pressure data of different dimensions；And to coefficient of variation CV given threshold, when the coefficient of variation is more than institute
During the threshold value of setting, then the data of the monitoring site are judged as invalid data, and deleted, be not involved in model calculating；
Step 3）Exceptional value judges；
Upper lower limit value is determined by triple standard difference method, i.e. normal value X is, determine higher limit
For, determine that lower limit is, reject for the value for not meeting this scope is exceptional value；
Step 4）Smoothed curve goes noise；
For each group data of monitoring point for removing exceptional value adopts least square fitting smoothed curve, it is first determined a letter
NumberApproach original function；If approximate function is, functional valueWith observationDifference be referred to as residual error,
Approximate function is weighed with residual errorQuality, concrete grammar is：
According to known data point, first with MATLAB solving equations, undetermined coefficient and fitting function are obtained；Recycle fitting function
Value replaces curve noise value, reaches the effect of curve smoothing；Further, can will replace fitting function value after noise value to enter again
Row fitting, repeat the above steps are until residual error meets required precision；
Step 5）Interpolation processing is carried out to missing values；
Row interpolation is entered to missing values using cubic spline function, the time series data repetition for processing monitoring is described by above-mentioned steps
The larger sequence data of value, missing values, exceptional value and dispersion；
When carrying out data processing in actual modeling process, the function that least square fitting goes out most to approach observation is first passed through,
The trend trend of general control data, while screening step 3）In fail by three times standard deviation send out remove exceptional value and pick
Remove, reduce the presence of error；
When model data is actually imported, the data of local segmentation are recycled, using spline interpolation method by missing values
And the part of abnormality value removing carries out polishing, to prevent the distortion of matched curve data, while remaining former rational observation.
2. the time series data cleaning method for pipe net modeling according to claim 1, it is characterised in that：Step 1）In,
Described time period data include the Monitoring Data of water factory's discharge pressure and water flow, Income gap, water-use model
Data, and pressure, the flow time series data of the different pipe network monitoring point positions for model checking.
3. the time series data cleaning method for pipe net modeling according to claim 1, it is characterised in that：Step 2）In,
The threshold value of the coefficient of variation may be set to 1, i.e. standard deviation sigma less than average value mu.
4. the time series data cleaning method for pipe net modeling according to claim 1, it is characterised in that：In step 4）
In, described functionCurve do not required all of data point on the graph, but need that data can be shown
Trend.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611011961.8A CN106649579A (en) | 2016-11-17 | 2016-11-17 | Time-series data cleaning method for pipe net modeling |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611011961.8A CN106649579A (en) | 2016-11-17 | 2016-11-17 | Time-series data cleaning method for pipe net modeling |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106649579A true CN106649579A (en) | 2017-05-10 |
Family
ID=58807594
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611011961.8A Pending CN106649579A (en) | 2016-11-17 | 2016-11-17 | Time-series data cleaning method for pipe net modeling |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106649579A (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563426A (en) * | 2017-08-25 | 2018-01-09 | 清华大学 | A kind of learning method of locomotive operation temporal aspect |
| CN108089938A (en) * | 2018-01-08 | 2018-05-29 | 湖南盈峰国创智能科技有限公司 | Method for processing abnormal data and device |
| CN108088916A (en) * | 2017-12-05 | 2018-05-29 | 国网山东省电力公司电力科学研究院 | A kind of method for improving and system of oil dissolved gas online monitoring data quality |
| CN109146311A (en) * | 2018-09-05 | 2019-01-04 | 武汉轻工大学 | Processing technology appraisal procedure, user equipment, storage medium and device |
| CN109146310A (en) * | 2018-09-05 | 2019-01-04 | 武汉轻工大学 | Processing efficiency appraisal procedure, equipment, storage medium and device |
| CN109325692A (en) * | 2018-09-27 | 2019-02-12 | 清华大学合肥公共安全研究院 | The data real-time analysis method and device of pipe network |
| CN109710893A (en) * | 2019-01-23 | 2019-05-03 | 江西理工大学 | It is a kind of for correcting the temporal-spatial interpolating method of Deformation Monitoring of Open Pit Mine abnormal data |
| CN109727070A (en) * | 2018-12-28 | 2019-05-07 | 中国银行股份有限公司 | A kind of determination method and device of potential any active ues |
| CN109740301A (en) * | 2019-03-14 | 2019-05-10 | 华北电力大学 | A kind of accounting method of the Gas Generator Set carbon emission amount based on BP neural network |
| CN109871458A (en) * | 2019-02-01 | 2019-06-11 | 南京泛在地理信息产业研究院有限公司 | A kind of street lamp special pipelines spatial data error calibration method based on comprehensive pipeline |
| CN110096497A (en) * | 2019-03-28 | 2019-08-06 | 中国农业科学院农业信息研究所 | A kind of agricultural output data intelligence cleaning method and system |
| CN110321934A (en) * | 2019-06-12 | 2019-10-11 | 深圳供电局有限公司 | A kind of method and system detecting user power utilization abnormal data |
| CN110413949A (en) * | 2019-08-02 | 2019-11-05 | 湖南联智桥隧技术有限公司 | A kind of data processing method in increasing or decreasing variation tendency |
| CN110648525A (en) * | 2019-09-25 | 2020-01-03 | 宁波三星医疗电气股份有限公司 | Data complementary reading method and device and power acquisition terminal |
| CN111352617A (en) * | 2020-03-16 | 2020-06-30 | 山东省物化探勘查院 | Magnetic method data auxiliary arrangement method based on Fortran language |
| CN112307410A (en) * | 2020-09-18 | 2021-02-02 | 天津大学 | Seawater temperature and salinity information time sequence prediction method based on shipborne CTD measurement data |
| CN112597144A (en) * | 2020-12-29 | 2021-04-02 | 农业农村部环境保护科研监测所 | Automatic cleaning method for production area environment monitoring data |
| CN112612824A (en) * | 2020-12-15 | 2021-04-06 | 重庆梅安森科技股份有限公司 | Water supply pipe network abnormal data detection method based on big data |
| CN112730319A (en) * | 2020-12-25 | 2021-04-30 | 北京领主科技有限公司 | Portable spectrometer |
| CN112818495A (en) * | 2021-02-22 | 2021-05-18 | 成都四为电子信息股份有限公司 | Novel dynamic correction method for pipeline pressure drop measurement and calculation algorithm parameters |
| CN112816366A (en) * | 2021-01-20 | 2021-05-18 | 武汉理工大学 | Method and system for selecting chemical reagent in asphalt material surface energy test |
| CN112883075A (en) * | 2021-01-22 | 2021-06-01 | 中国地质环境监测院(自然资源部地质灾害技术指导中心) | Landslide universal type ground surface displacement monitoring data missing and abnormal value processing method |
| CN112905960A (en) * | 2021-02-18 | 2021-06-04 | 中国环境科学研究院 | Method and product for judging validity of tail gas plume data and screening tail gas plume data |
| CN112948204A (en) * | 2021-02-07 | 2021-06-11 | 上海汉询软件有限公司 | Data processing system based on DataRobot technology |
| CN113722651A (en) * | 2021-07-13 | 2021-11-30 | 山西三友和智慧信息技术股份有限公司 | Comparison method for multi-data correction smoothness based on NIR high-dimensional data |
| CN113901074A (en) * | 2021-09-26 | 2022-01-07 | 广州虎牙科技有限公司 | Abnormality determination method and apparatus for ad hoc query, electronic device, and medium |
| CN114064760A (en) * | 2021-11-18 | 2022-02-18 | 广州泰禾大数据服务有限公司 | Multi-dimensional early warning analysis and judgment method for data |
| CN114124340A (en) * | 2020-08-28 | 2022-03-01 | 京东方科技集团股份有限公司 | Time synchronization method and device for nodes of Internet of things, storage medium and electronic equipment |
| CN114968990A (en) * | 2022-04-12 | 2022-08-30 | 青岛沃柏斯智能实验科技有限公司 | Design method of diagnosis model for influencing factors of experimental data |
| CN114996318A (en) * | 2022-07-12 | 2022-09-02 | 成都唐源电气股份有限公司 | Automatic judgment method and system for processing mode of abnormal value of detection data |
| CN116304939A (en) * | 2023-02-22 | 2023-06-23 | 中广核智能科技(深圳)有限责任公司 | Multi-monitoring data evolution rule identification method and system based on waveform alignment |
| CN116610538A (en) * | 2023-07-21 | 2023-08-18 | 合肥喆塔科技有限公司 | Trending equipment parameter management and control method, system, equipment and storage medium |
| CN117112999A (en) * | 2023-07-24 | 2023-11-24 | 西南交通大学 | Drilling parameter standardized cleaning method and device based on dynamic linear piecewise representation |
| CN117148784A (en) * | 2023-10-27 | 2023-12-01 | 山东山森数控技术有限公司 | Operation fault analysis method for multi-axis multi-channel numerical control system |
| CN117436153A (en) * | 2023-12-15 | 2024-01-23 | 华能济南黄台发电有限公司 | Entity characteristic parameterization implementation method applied to power plant heat supply |
| CN117786281A (en) * | 2024-02-23 | 2024-03-29 | 中国海洋大学 | Optimization calculation method for deposition rate and error of deposit columnar sample |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102339288A (en) * | 2010-07-21 | 2012-02-01 | 中国移动通信集团辽宁有限公司 | Method and device for detecting abnormal data of data warehouse |
| CN102360378A (en) * | 2011-10-10 | 2012-02-22 | 南京大学 | Outlier detection method for time-series data |
| CN104537034A (en) * | 2014-12-22 | 2015-04-22 | 国家电网公司 | Electric-transmission-and-transformation-equipment state monitoring data cleaning method based on time series analysis |
| CN104679834A (en) * | 2015-02-06 | 2015-06-03 | 北京工商大学 | Time series data cleaning method and system |
-
2016
- 2016-11-17 CN CN201611011961.8A patent/CN106649579A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102339288A (en) * | 2010-07-21 | 2012-02-01 | 中国移动通信集团辽宁有限公司 | Method and device for detecting abnormal data of data warehouse |
| CN102360378A (en) * | 2011-10-10 | 2012-02-22 | 南京大学 | Outlier detection method for time-series data |
| CN104537034A (en) * | 2014-12-22 | 2015-04-22 | 国家电网公司 | Electric-transmission-and-transformation-equipment state monitoring data cleaning method based on time series analysis |
| CN104679834A (en) * | 2015-02-06 | 2015-06-03 | 北京工商大学 | Time series data cleaning method and system |
Non-Patent Citations (1)
| Title |
|---|
| 宋金玉 等: "数据质量及数据清洗方法", 《指挥信息系统与技术》 * |
Cited By (50)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563426A (en) * | 2017-08-25 | 2018-01-09 | 清华大学 | A kind of learning method of locomotive operation temporal aspect |
| CN108088916B (en) * | 2017-12-05 | 2019-06-04 | 国网山东省电力公司电力科学研究院 | A kind of method for improving and system of oil dissolved gas online monitoring data quality |
| CN108088916A (en) * | 2017-12-05 | 2018-05-29 | 国网山东省电力公司电力科学研究院 | A kind of method for improving and system of oil dissolved gas online monitoring data quality |
| CN108089938A (en) * | 2018-01-08 | 2018-05-29 | 湖南盈峰国创智能科技有限公司 | Method for processing abnormal data and device |
| CN108089938B (en) * | 2018-01-08 | 2021-04-09 | 湖南盈峰国创智能科技有限公司 | Abnormal data processing method and device |
| CN109146311A (en) * | 2018-09-05 | 2019-01-04 | 武汉轻工大学 | Processing technology appraisal procedure, user equipment, storage medium and device |
| CN109146310A (en) * | 2018-09-05 | 2019-01-04 | 武汉轻工大学 | Processing efficiency appraisal procedure, equipment, storage medium and device |
| CN109325692A (en) * | 2018-09-27 | 2019-02-12 | 清华大学合肥公共安全研究院 | The data real-time analysis method and device of pipe network |
| CN109727070A (en) * | 2018-12-28 | 2019-05-07 | 中国银行股份有限公司 | A kind of determination method and device of potential any active ues |
| CN109727070B (en) * | 2018-12-28 | 2021-11-19 | 中国银行股份有限公司 | Method and device for determining potential active users |
| CN109710893A (en) * | 2019-01-23 | 2019-05-03 | 江西理工大学 | It is a kind of for correcting the temporal-spatial interpolating method of Deformation Monitoring of Open Pit Mine abnormal data |
| CN109710893B (en) * | 2019-01-23 | 2023-04-07 | 江西理工大学 | Time-space interpolation method for correcting abnormal data of mine slope deformation monitoring |
| CN109871458A (en) * | 2019-02-01 | 2019-06-11 | 南京泛在地理信息产业研究院有限公司 | A kind of street lamp special pipelines spatial data error calibration method based on comprehensive pipeline |
| CN109740301A (en) * | 2019-03-14 | 2019-05-10 | 华北电力大学 | A kind of accounting method of the Gas Generator Set carbon emission amount based on BP neural network |
| CN110096497A (en) * | 2019-03-28 | 2019-08-06 | 中国农业科学院农业信息研究所 | A kind of agricultural output data intelligence cleaning method and system |
| CN110321934A (en) * | 2019-06-12 | 2019-10-11 | 深圳供电局有限公司 | A kind of method and system detecting user power utilization abnormal data |
| CN110413949A (en) * | 2019-08-02 | 2019-11-05 | 湖南联智桥隧技术有限公司 | A kind of data processing method in increasing or decreasing variation tendency |
| CN110413949B (en) * | 2019-08-02 | 2021-03-09 | 湖南联智科技股份有限公司 | Data processing method with increasing or decreasing trend |
| CN110648525B (en) * | 2019-09-25 | 2020-07-28 | 宁波三星医疗电气股份有限公司 | Data complementary reading method and device and power acquisition terminal |
| CN110648525A (en) * | 2019-09-25 | 2020-01-03 | 宁波三星医疗电气股份有限公司 | Data complementary reading method and device and power acquisition terminal |
| CN111352617A (en) * | 2020-03-16 | 2020-06-30 | 山东省物化探勘查院 | Magnetic method data auxiliary arrangement method based on Fortran language |
| CN114124340A (en) * | 2020-08-28 | 2022-03-01 | 京东方科技集团股份有限公司 | Time synchronization method and device for nodes of Internet of things, storage medium and electronic equipment |
| CN112307410A (en) * | 2020-09-18 | 2021-02-02 | 天津大学 | Seawater temperature and salinity information time sequence prediction method based on shipborne CTD measurement data |
| CN112612824A (en) * | 2020-12-15 | 2021-04-06 | 重庆梅安森科技股份有限公司 | Water supply pipe network abnormal data detection method based on big data |
| CN112730319A (en) * | 2020-12-25 | 2021-04-30 | 北京领主科技有限公司 | Portable spectrometer |
| CN112597144A (en) * | 2020-12-29 | 2021-04-02 | 农业农村部环境保护科研监测所 | Automatic cleaning method for production area environment monitoring data |
| CN112597144B (en) * | 2020-12-29 | 2022-11-08 | 农业农村部环境保护科研监测所 | Automatic cleaning method for production place environment monitoring data |
| CN112816366A (en) * | 2021-01-20 | 2021-05-18 | 武汉理工大学 | Method and system for selecting chemical reagent in asphalt material surface energy test |
| CN112883075A (en) * | 2021-01-22 | 2021-06-01 | 中国地质环境监测院(自然资源部地质灾害技术指导中心) | Landslide universal type ground surface displacement monitoring data missing and abnormal value processing method |
| CN112883075B (en) * | 2021-01-22 | 2024-04-05 | 中国地质环境监测院(自然资源部地质灾害技术指导中心) | Landslide universal type ground surface displacement monitoring data missing and outlier processing method |
| CN112948204A (en) * | 2021-02-07 | 2021-06-11 | 上海汉询软件有限公司 | Data processing system based on DataRobot technology |
| CN112905960A (en) * | 2021-02-18 | 2021-06-04 | 中国环境科学研究院 | Method and product for judging validity of tail gas plume data and screening tail gas plume data |
| CN112905960B (en) * | 2021-02-18 | 2024-04-19 | 中国环境科学研究院 | Method and product for judging effectiveness of tail gas smoke plume data and screening tail gas smoke plume data |
| CN112818495A (en) * | 2021-02-22 | 2021-05-18 | 成都四为电子信息股份有限公司 | Novel dynamic correction method for pipeline pressure drop measurement and calculation algorithm parameters |
| CN113722651A (en) * | 2021-07-13 | 2021-11-30 | 山西三友和智慧信息技术股份有限公司 | Comparison method for multi-data correction smoothness based on NIR high-dimensional data |
| CN113901074A (en) * | 2021-09-26 | 2022-01-07 | 广州虎牙科技有限公司 | Abnormality determination method and apparatus for ad hoc query, electronic device, and medium |
| CN114064760A (en) * | 2021-11-18 | 2022-02-18 | 广州泰禾大数据服务有限公司 | Multi-dimensional early warning analysis and judgment method for data |
| CN114064760B (en) * | 2021-11-18 | 2022-12-13 | 广州泰禾大数据服务有限公司 | Multi-dimensional early warning analysis and judgment method for data |
| CN114968990A (en) * | 2022-04-12 | 2022-08-30 | 青岛沃柏斯智能实验科技有限公司 | Design method of diagnosis model for influencing factors of experimental data |
| CN114996318A (en) * | 2022-07-12 | 2022-09-02 | 成都唐源电气股份有限公司 | Automatic judgment method and system for processing mode of abnormal value of detection data |
| CN114996318B (en) * | 2022-07-12 | 2022-11-04 | 成都唐源电气股份有限公司 | Automatic judgment method and system for processing mode of abnormal value of detection data |
| CN116304939A (en) * | 2023-02-22 | 2023-06-23 | 中广核智能科技(深圳)有限责任公司 | Multi-monitoring data evolution rule identification method and system based on waveform alignment |
| CN116610538A (en) * | 2023-07-21 | 2023-08-18 | 合肥喆塔科技有限公司 | Trending equipment parameter management and control method, system, equipment and storage medium |
| CN117112999B (en) * | 2023-07-24 | 2024-03-29 | 西南交通大学 | Drilling parameter standardized cleaning method and device based on dynamic linear piecewise representation |
| CN117112999A (en) * | 2023-07-24 | 2023-11-24 | 西南交通大学 | Drilling parameter standardized cleaning method and device based on dynamic linear piecewise representation |
| CN117148784A (en) * | 2023-10-27 | 2023-12-01 | 山东山森数控技术有限公司 | Operation fault analysis method for multi-axis multi-channel numerical control system |
| CN117148784B (en) * | 2023-10-27 | 2024-01-26 | 山东山森数控技术有限公司 | Operation fault analysis method for multi-axis multi-channel numerical control system |
| CN117436153A (en) * | 2023-12-15 | 2024-01-23 | 华能济南黄台发电有限公司 | Entity characteristic parameterization implementation method applied to power plant heat supply |
| CN117436153B (en) * | 2023-12-15 | 2024-03-08 | 华能济南黄台发电有限公司 | Entity characteristic parameterization implementation method applied to power plant heat supply |
| CN117786281A (en) * | 2024-02-23 | 2024-03-29 | 中国海洋大学 | Optimization calculation method for deposition rate and error of deposit columnar sample |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106649579A (en) | Time-series data cleaning method for pipe net modeling | |
| CN110992354B (en) | Abnormal region detection method for countering self-encoder based on introduction of automatic memory mechanism | |
| CN110134919B (en) | Method for cleaning abnormal data of wind turbine generator | |
| CN102332011B (en) | Method for selecting effective data of in-orbit spacecraft | |
| CN107292429A (en) | A kind of Demand Forecast Model system of selection analyzed based on demand characteristics | |
| CN111275307A (en) | Quality control method for high-frequency continuous observation data of automatic online water quality station | |
| CN109815855B (en) | Electronic equipment automatic test method and system based on machine learning | |
| Hwang et al. | Detection and correction method of erroneous data using quantile pattern and LSTM | |
| CN110083803A (en) | Based on Time Series AR IMA model water intaking method for detecting abnormality and system | |
| CN113130014A (en) | Rare earth extraction simulation method and system based on multi-branch neural network | |
| CN112818495A (en) | Novel dynamic correction method for pipeline pressure drop measurement and calculation algorithm parameters | |
| Goswami et al. | Comparative assessment of six automatic optimization techniques for calibration of a conceptual rainfall—runoff model | |
| CN106357458B (en) | Network element method for detecting abnormality and device | |
| CN111198979A (en) | Method and system for cleaning big data for power transmission and transformation reliability evaluation | |
| CN113094826A (en) | Task reliability-based remaining life prediction method for multi-state manufacturing system | |
| CN108090241A (en) | trend variable identification method and system of continuous process | |
| CN111679970A (en) | Robot software system running environment state prediction method | |
| Amiri et al. | Identifying time of a monotonic change in the fraction nonconforming of a high-quality process | |
| CN110907135A (en) | Control method and device of manufacturing equipment | |
| CN112733281B (en) | Machine tool reliability assessment method considering truncated data deletion | |
| CN109800866B (en) | Reliability increase prediction method based on GA-Elman neural network | |
| CN110348005A (en) | Distribution net equipment status data processing method, device, computer equipment and medium | |
| CN115526428A (en) | Drainage pipe network sewage flow prediction method based on space-time diagram convolutional network | |
| CN105842535B (en) | A kind of main syndrome screening technique of harmonic wave based on similar features fusion | |
| CN114298413A (en) | Hydroelectric generating set runout trend prediction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170510 |
|
| WD01 | Invention patent application deemed withdrawn after publication |