CN108667688A - A kind of data monitoring method and device - Google Patents
A kind of data monitoring method and device Download PDFInfo
- Publication number
- CN108667688A CN108667688A CN201810353780.6A CN201810353780A CN108667688A CN 108667688 A CN108667688 A CN 108667688A CN 201810353780 A CN201810353780 A CN 201810353780A CN 108667688 A CN108667688 A CN 108667688A
- Authority
- CN
- China
- Prior art keywords
- data
- acquisition
- comparison
- monitored
- moment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
- H04L43/045—Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L67/125—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Environmental & Geological Engineering (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of data monitoring method and devices, by the data information for acquiring data source to be monitored, at the time of data information includes the data and gathered data of acquisition, obtain the comparison data of data source to be monitored, historical data at the time of comparison data is gathered data, data wave rotating ring ratio is determined according to the data of acquisition and comparison data, when data wave rotating ring ratio is more than first threshold, it determines that the data of the data source to be monitored of acquisition are abnormal data, generates warning information.This programme is by the way that directly the data of acquisition are compared with comparison data, when data wave rotating ring ratio is more than first threshold, generate warning information, wherein, it is handled without the data to acquisition, comparison process is simple to operation, without complicated data analysis process, the algorithm for simplifying data monitoring, has saved data processing amount, improves the fluency of system operation and improves the monitoring scope of application using general-purpose algorithm.
Description
Technical field
The present invention relates to monitoring field more particularly to a kind of data monitoring methods and device.
Background technology
In informationization is runed, the mechanisms such as government, tissue, company all can in real time give birth to there are many software or internet of things equipment
At mass data.
However, to realize the early warning to the data of acquisition, usually it is required for using more complicated algorithm.Such as：Application
Number patent application for being 201410495326.6, the GPS positioning for utilizing smart mobile phone included and gyro sensor, obtain vehicle
Then acceleration in driving procedure and steering wheel angle data were added using wavelet transformation from storage extracting data every 10 seconds
1 normalized energy of db5 wavelet scales, the turning angle of steering wheel velocity standard of speed are poor, wheel steering db5 wavelet scales 4 are returned
One changes energy as degree of fatigue Judging index.Warning algorithm is complicated in the program, needs to occupy larger data processing amount, drops
The fluency of low system operation.
Invention content
In view of this, a kind of data monitoring method of present invention offer and device, to solve to realize to adopting in the prior art
The early warning of the data of collection needs more complicated algorithm, occupies larger data processing amount, reduces asking for system operation fluency
Topic, concrete scheme are as follows：
A kind of data monitoring method, including：
The data information of data source to be monitored is acquired, the data information includes：The data source to be monitored of acquisition
At the time of data and the acquisition data；
The comparison data of the data source to be monitored is obtained, at the time of the comparison data is the acquisition data
Historical data；
Data wave rotating ring ratio is determined according to the data of the data source to be monitored of the acquisition and the comparison data；
When the data wave rotating ring ratio is more than first threshold, the data of the data source to be monitored of acquisition are abnormal number
According to generation warning information.
Further, the comparison data for obtaining the data source to be monitored, the comparison data is the acquisition institute
Historical data at the time of data is stated, including：
The comparison data information of the data source to be monitored is obtained, the comparison data information includes：The number to be monitored
According to source comparison data and acquire the comparison data at the time of；
Corresponding acquisition is described at the time of determination with the acquisition data from the comparison data information compares logarithm
According at the time of be the first moment, obtain the comparison data of first moment acquisition.
Further, further include：
Sequencing at the time of according to the acquisition comparison data, which generates, compares chart；
Data drawing list is generated at the time of according to the acquisition data；
Determine that Future Data acquires the pre- of moment according to the data variation trend of the comparison chart and the data drawing list
Measured data trend.
Further, the comparison data for obtaining the data source to be monitored, the comparison data is the acquisition institute
Historical data at the time of data is stated, including：
Determine the data collection cycle of the data source to be monitored；
Obtain the historical data information of the predetermined period number acquired before the Current data acquisition period, the predetermined period
Do not include abnormal data information in the historical data information of number；
It determines the average value of the historical data of the predetermined period number, obtains comparison data information；
Corresponding acquisition is described at the time of determination with the acquisition data from the comparison data information compares logarithm
According at the time of be the first moment, obtain the comparison data of first moment acquisition.
Further, the data of the data source to be monitored according to the acquisition and the comparison data determine number
According to roll-collar ratio, including：
The data of the acquisition and the comparison data are made the difference to obtain the first difference, by first difference and the ratio
Quotient is done to data, obtains data wave rotating ring ratio.
Further, further include：
If in the same data collection cycle, the data wave rotating ring ratio at current time is less than first threshold, and, it is previous
The data wave rotating ring ratio at moment is more than first threshold, cancels the warning information.
A kind of data monitoring device, including：Collecting unit, acquiring unit, determination unit and generation unit, wherein：
The collecting unit is used to acquire the data information of data source to be monitored, and the data information includes：The institute of acquisition
At the time of stating the data of data source to be monitored and acquire the data；
The acquiring unit is used to obtain the comparison data of the data source to be monitored, and the comparison data is the acquisition
The historical data at the time of data；
The determination unit is used for true according to the data of the data source to be monitored of the acquisition and the comparison data
Fixed number is according to roll-collar ratio；
The generation unit is used for when the data wave rotating ring ratio is more than first threshold, the data to be monitored of acquisition
The data in source are abnormal data, generate warning information.
Further, the acquiring unit is used for：
The comparison data information of the data source to be monitored is obtained, the comparison data information includes：The number to be monitored
According to source comparison data and acquire the comparison data at the time of；From the comparison data information described in the determining and acquisition
It was the first moment at the time of the corresponding acquisition comparison data at the time of data, obtains the ratio logarithm of the first moment acquisition
According to.
Further, further include：Predicting unit, wherein：
The sequencing at the time of predicting unit is used for according to the acquisition comparison data, which generates, compares chart；
Data drawing list is generated at the time of according to the acquisition data；Become according to the data of the comparison chart and the data drawing list
Change trend determines the prediction data trend at Future Data acquisition moment.
Further, the acquiring unit is used for：
Determine the data collection cycle of the data source to be monitored；What is acquired before obtaining the Current data acquisition period makes a reservation for
The historical data information of cycle times does not include abnormal data information in the historical data information of the predetermined period number；Really
The average value of the historical data of the fixed predetermined period number, obtains comparison data information；From the comparison data information really
It is fixed be the first moment, acquisition described first at the time of the corresponding acquisition comparison data at the time of acquisition data
The comparison data of moment acquisition.
It can be seen from the above technical proposal that data monitoring method disclosed in the present application and device, to be monitored by acquiring
The data information of data source obtains the ratio of data source to be monitored at the time of data information includes the data and gathered data of acquisition
To data, historical data at the time of comparison data is gathered data determines data wave according to the data of acquisition and comparison data
Rotating ring ratio determines that the data of the data source to be monitored of acquisition are abnormal data when data wave rotating ring ratio is more than first threshold, raw
At warning information.This programme is by the way that directly the data of acquisition are compared with comparison data, when data wave rotating ring ratio is more than the
When one threshold value, that is, generate warning information, wherein handled without the data to acquisition, comparison process is simple to operation, is not necessarily to
Complicated data analysis process, simplifies the algorithm of data monitoring, has saved data processing amount, improve the smoothness of system operation
It spends and the monitoring scope of application is improved using general-purpose algorithm.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
Obtain other attached drawings according to these attached drawings.
Fig. 1 is a kind of flow chart of data monitoring method disclosed by the embodiments of the present invention；
Fig. 2 is a kind of flow chart of data monitoring method disclosed by the embodiments of the present invention；
Fig. 3 is a kind of flow chart of data monitoring method disclosed by the embodiments of the present invention；
Fig. 4 is a kind of structural schematic diagram of data monitoring device disclosed by the embodiments of the present invention；
Fig. 5 is comparison chart disclosed by the embodiments of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
The invention discloses a kind of data monitoring method, flow chart is as shown in Figure 1, include：
Step S11, the data information of data source to be monitored is acquired, data information includes：The data source to be monitored of acquisition
Data and at the time of acquire the data；
Acquiring the data of data source to be monitored can be specially：By the way that adapter distributed deployment will be acquired in number to be monitored
According on source, by Kafka message-oriented middlewares, it will acquire and focused on the collected information taken to processor of adapter.
Wherein, Kafka is a kind of distributed post subscription message system of high-throughput, can be handled in extensive website
Everything flow data, support online horizontal extension, high-performance, such as：Message duration time complexity is O (1), single machine branch
The transmission of 100K message above per second is held, supports TB grades of size of messages.The transmission of message is safe and reliable, and service has stable fault-tolerant
Ability, no Single Point of Faliure can be to the big datas technology slitless connection such as Hadoop.
Adapter is acquired using software programming come the acquisition of adaptation data, the data for only treating monitoring data source carry out on time
Between line simple collection or convergence, the change data without the specific business according to data source to be monitored do not make at complex transformations
Reason is to reduce data processing amount.Wherein, acquisition adapter may be by different hardware realizations, such as：GPS locator, again
The sensors such as power gyroscope.
Further, after the data information for acquiring data source to be monitored, it is necessary first to by collected data information
It is stored, to be compared as historical data information.
Specifically, will particularly which data collection cycle, and be which of data collection cycle number
It is stored to memory or database according to acquisition moment and data acquisition moment institute collected data, so as to as going through
When history data, can specify which data acquisition moment which data collection cycle is the data be is acquired.
Step S12, the comparison data of data source to be monitored is obtained, comparison data is history number at the time of acquiring the data
According to；
Comparison data can be：Pre-stored normal data, or：Historical data.Wherein, historical data can be with
For：The data in the normal data collection cycle of any data before the Current data acquisition period, or：Current
The average value of data in predetermined quantity data collection cycle before data collection cycle.
Specifically, determining the data collection cycle of data source to be monitored, acquired before obtaining the Current data acquisition period
The historical data information of predetermined period number does not include abnormal data information in the historical data information of predetermined period number, really
Determine the average value of the historical data of predetermined period number, obtains comparison data information.
Such as：
One data collection cycle is 24 hours, and predetermined period number is 7 times, and the Current data acquisition period is December 10
Day, i.e. comparison data information is：7 days before December 10, i.e., the historical data information that December 3 to December 9 acquires is put down
Mean value, wherein it is normal data in the historical data information that December 3 to December 9 acquires, i.e., does not include abnormal data,
If including abnormal data, need to calculate average value again after excluding the abnormal data.
Further, can also include：Corresponding acquisition at the time of determination from comparison data information with the acquisition data
It was the first moment at the time of comparison data, obtains the comparison data of the first moment acquisition.
Wherein, at the time of comparison data information includes the comparison data and acquisition comparison data of data source to be monitored.
Specifically, can be：The ratio corresponding to each data acquisition moment in a data collection cycle is obtained in advance
To data, later, the comparison data corresponding to the moment is acquired after the currently acquisition moment collects data, then from each data
Middle determination data corresponding with the current acquisition moment acquire the moment, later, determine that the data acquisition moment corresponding compares logarithm
According to.This is just needed after getting the comparison data in a data collection cycle corresponding to each data acquisition moment,
The corresponding data of each comparison data are acquired into moment corresponding label, i.e. comparison data information, comparison data information
In not only include comparison data, further include the acquisition moment corresponding with the comparison data.
Can also be：It only obtains data corresponding with the current acquisition moment and acquires moment corresponding comparison data.Working as
After the preceding acquisition moment collects data, then determine that one or more data corresponding with the current acquisition moment are adopted from historical data
Collect moment corresponding data, however, it is determined that be multiple data, then in different data collection cycles with the current acquisition moment pair
This multiple data is done average value, generates comparison data later by the data for the data acquisition moment acquisition answered.
Step S13, data wave rotating ring ratio is determined according to the data of the data source to be monitored of acquisition and comparison data；
Determine that data wave rotating ring ratio is current in order to determine according to the data of the data source to be monitored of acquisition and comparison data
In data collection cycle, whether the fluctuation range between the data and comparison data of current time acquisition exceeds first threshold, when
When it exceeds first threshold, shows that the data are abnormal data, warning information is generated, to notify operating personnel to number to be monitored
It is handled according to source；When it is without departing from first threshold, show the data compared with comparison data, fluctuating range is in the first threshold
In value, belong to normal fluctuation range, which is that normal data is normally believed without sending out any information alternatively, only sending out data
Breath.
Further, while memory or database purchase have collected data, memory or database may be used also
To be stored with the data information for sending out warning information, such as：Some data acquisition moment in some data collection cycle adopts
The data collected have been issued warning information, then the data be abnormal data, using the data in memory or database as
When historical data participates in comparing, the abnormal data is excluded, avoids it from participating in comparing as comparison data, this reduces history
Data are as data error caused by comparison data.
Further, can also be：After having handled the data at current time, wherein handle the data at current time
May include：Abnormal data stores after sending out warning information, alternatively, normal data does not send out warning information, is only stored.
Later, from searching in the Current data acquisition period whether also have other untreated complete data in database, i.e., in current data
In collection period whether it is in need send out warning information, but since warning information caused by equipment shutdown or other reasons is not sent out
The case where, if so, handling the warning information, the warning information is sent out, also, need to include data in the warning information
Acquire the moment.
Step S14, when data wave rotating ring ratio is more than first threshold, determine that the data of the data source to be monitored of acquisition are different
Regular data generates warning information.
In data monitoring method disclosed in the present embodiment, can data acquisition be carried out to multiple equipment or system simultaneously,
Specifically, the monitoring to data can be realized by Browser/Server (Browser/Server Mode) program architecture,
In, Browser/Server can across multiple terminals displaying, such as：The ends PC, Phone, Pad etc. only need smart machine to provide browser, nothing
Client software packet need to be provided to equipment various terminals, and it is possible to provide restful interfaces based on http agreements, it is convenient with
Other systems are extended or are integrated.
Data monitoring method disclosed in the present embodiment, by acquiring the data information of data source to be monitored, data packets
At the time of including the data and gathered data of acquisition, the comparison data of data source to be monitored is obtained, comparison data is gathered data
The historical data at moment determines data wave rotating ring ratio according to the data of acquisition and comparison data, when data wave rotating ring ratio is more than the
When one threshold value, determines that the data of the data source to be monitored of acquisition are abnormal data, generate warning information.This programme will be by directly will
The data of acquisition are compared with comparison data, when data wave rotating ring ratio is more than first threshold, that is, generate warning information,
In, it is handled without the data to acquisition, comparison process is simple to operation, without complicated data analysis process, simplifies
The algorithm of data monitoring, has saved data processing amount, improves the fluency of system operation.
Present embodiment discloses a kind of data monitoring method, flow chart is as shown in Fig. 2, include：
Step S21, the data information of data source to be monitored is acquired, data information includes：The data source to be monitored of acquisition
Data and at the time of acquire the data；
Step S22, the comparison data information of data source to be monitored is obtained, comparison data information includes：Data source to be monitored
Comparison data and at the time of acquire the comparison data；
Step S23, from comparison data information determine with acquire the data at the time of it is corresponding acquire comparison data when
It was the first moment to carve, and obtained the comparison data of the first moment acquisition；
Step S24, according to acquisition comparison data sequencing at the time of, which generates, compares chart；
Step S25, according to gathered data data drawing list is generated at the time of；
Step S26, determine that Future Data acquires the pre- of moment according to the data variation trend for comparing chart and data drawing list
Measured data trend.
Sequencing at the time of according to acquisition comparison data, which generates, compares chart, that is, obtains data collection cycle
Comparison data, the wherein comparison data of a data collection cycle can be the predetermined number in the Current data acquisition period
The average value of each data acquisition moment gathered data, the average value that each data are acquired to the moment form in data collection cycle
The comparison data of one complete data collection cycle.
The comparison data of one data collection cycle is acquired into the sequencing at moment according to corresponding data and generates ratio
To chart, specifically, the comparison chart can be a curve, in order to check the fluctuating range of its data.
Data drawing list is generated at the time of according to gathered data, can be specially：Within the Current data acquisition period, Mei Daoyi
A data acquire the moment with regard to gathered data, and the data are acquired moment and its corresponding data composition Current data acquisition period
A part in corresponding chart.
Data drawing list is generated at the time of according to gathered data, can be specially：It is every within the Current data acquisition period
One data acquires the moment with regard to gathered data, and the data are acquired moment and its corresponding data composition Current data acquisition
The part in chart corresponding to period.
Specifically, can be as shown in Figure 5：
Darker curve is different data acquisition moment corresponding data in the Current data acquisition period, and lighter curve is advance
The comparison data of the complete data collection cycle obtained.
By the above chart it was determined that different data acquire the moment, the data in the Current data acquisition period with compare
The fluctuation size between the data in curve corresponding to data, and can be acquired according to multiple data before current in the moment
Data collection cycle moves towards to determine in the Current data acquisition period Current data acquisition moment with the curve in comparison data curve
The prediction data at one or several data acquisition moment in future later moves towards trend, and prediction future time instance data are extracted to realize
Trend.
Further, can also include：Data wave is determined according to the data of the data source to be monitored of acquisition and comparison data
Rotating ring ratio determines that the data of the data source to be monitored of acquisition are abnormal data when data wave rotating ring ratio is more than first threshold, raw
At warning information.
Data monitoring method disclosed in the present embodiment, by acquiring the data information of data source to be monitored, data packets
At the time of including the data and gathered data of acquisition, the comparison data of data source to be monitored is obtained, comparison data is gathered data
The historical data at moment determines data wave rotating ring ratio according to the data of acquisition and comparison data, when data wave rotating ring ratio is more than the
When one threshold value, determines that the data of the data source to be monitored of acquisition are abnormal data, generate warning information.This programme will be by directly will
The data of acquisition are compared with comparison data, when data wave rotating ring ratio is more than first threshold, that is, generate warning information,
In, it is handled without the data to acquisition, comparison process is simple to operation, without complicated data analysis process, simplifies
The algorithm of data monitoring, has saved data processing amount, improves the fluency of system operation.
Present embodiment discloses a kind of data monitoring method, flow chart is as shown in figure 3, include：
Step S31, the data information of data source to be monitored is acquired, data information includes：The data source to be monitored of acquisition
Data and at the time of acquire the data；
Step S32, the comparison data of data source to be monitored, history number at the time of comparison data is gathered data are obtained
According to；
Step S33, the data of acquisition and comparison data are made the difference to obtain the first difference, the first difference is done with comparison data
Quotient obtains data wave rotating ring ratio；
Specifically, if Current data acquisition moment collected data are A in the Current data acquisition period, history compares logarithm
Corresponding comparison data is B at the time of corresponding with the Current data acquisition moment in, then, data wave rotating ring ratio C is：
C=| A-B/B |.
When C is more than first threshold, show the collected number of Current data acquisition moment institute in the Current data acquisition period
According to for abnormal data, needing to generate warning information.
Further, when C is less than first threshold, when detecting that last data acquires in the Current data acquisition period
It is abnormal data to carve the collected data of institute, and corresponding data wave rotating ring ratio is more than first threshold, due to Current data acquisition
The roll-collar ratio of moment collected data is less than first threshold, therefore, the alarm that the revocation last data acquisition moment generates
Information, to prompt staff's data to restore normal range (NR).
Step S34, when data wave rotating ring ratio is more than first threshold, determine that the data of the data source to be monitored of acquisition are different
Regular data generates warning information.
Data monitoring method disclosed in the present embodiment, by acquiring the data information of data source to be monitored, data packets
At the time of including the data and gathered data of acquisition, the comparison data of data source to be monitored is obtained, comparison data is gathered data
The historical data at moment determines data wave rotating ring ratio according to the data of acquisition and comparison data, when data wave rotating ring ratio is more than the
When one threshold value, determines that the data of the data source to be monitored of acquisition are abnormal data, generate warning information.This programme will be by directly will
The data of acquisition are compared with comparison data, when data wave rotating ring ratio is more than first threshold, that is, generate warning information,
In, it is handled without the data to acquisition, comparison process is simple to operation, without complicated data analysis process, simplifies
The algorithm of data monitoring, has saved data processing amount, improves the fluency of system operation.
Present embodiment discloses a kind of data monitoring devices, and structural schematic diagram is as shown in figure 4, include：
Collecting unit 41, acquiring unit 42, determination unit 43 and generation unit 44.
Wherein, collecting unit 41 is used to acquire the data information of data source to be monitored, and data information includes：Acquisition waits supervising
At the time of controlling the data of data source and acquire the data；
Acquiring the data of data source to be monitored can be specially：By the way that adapter distributed deployment will be acquired in number to be monitored
According on source, by Kafka message-oriented middlewares, it will acquire and focused on the collected information taken to processor of adapter.
Wherein, Kafka is a kind of distributed post subscription message system of high-throughput, can be handled in extensive website
Everything flow data, support online horizontal extension, high-performance, such as：Message duration time complexity is O (1), single machine branch
The transmission of 100K message above per second is held, supports TB grades of size of messages.The transmission of message is safe and reliable, and service has stable fault-tolerant
Ability, no Single Point of Faliure can be to the big datas technology slitless connection such as Hadoop.
Adapter is acquired using software programming come the acquisition of adaptation data, the data for only treating monitoring data source carry out on time
Between line simple collection or convergence, the change data without the specific business according to data source to be monitored do not make at complex transformations
Reason is to reduce data processing amount.Wherein, acquisition adapter may be by different hardware realizations, such as：GPS locator, again
The sensors such as power gyroscope.
Further, after the data information for acquiring data source to be monitored, it is necessary first to by collected data information
It is stored, to be compared as historical data information.
Specifically, will particularly which data collection cycle, and be which of data collection cycle number
It is stored to memory or database according to acquisition moment and data acquisition moment institute collected data, so as to as going through
When history data, can specify which data acquisition moment which data collection cycle is the data be is acquired.
Acquiring unit 42 is used to obtain the comparison data of data source to be monitored, going through at the time of comparison data is gathered data
History data；
Comparison data can be：Pre-stored normal data, or：Historical data.Wherein, historical data can be with
For：The data in the normal data collection cycle of any data before the Current data acquisition period, or：Current
The average value of data in predetermined quantity data collection cycle before data collection cycle.
Specifically, determining the data collection cycle of data source to be monitored, acquired before obtaining the Current data acquisition period
The historical data information of predetermined period number does not include abnormal data information in the historical data information of predetermined period number, really
Determine the average value of the historical data of predetermined period number, obtains comparison data information.
Such as：
One data collection cycle is 24 hours, and predetermined period number is 7 times, and the Current data acquisition period is December 10
Day, i.e. comparison data information is：7 days before December 10, i.e., the historical data information that December 3 to December 9 acquires is put down
Mean value, wherein it is normal data in the historical data information that December 3 to December 9 acquires, i.e., does not include abnormal data,
If including abnormal data, need to calculate average value again after excluding the abnormal data.
Further, can also include：Corresponding acquisition at the time of determination from comparison data information with the acquisition data
It was the first moment at the time of comparison data, obtains the comparison data of the first moment acquisition.
Wherein, at the time of comparison data information includes the comparison data and acquisition comparison data of data source to be monitored.
Specifically, can be：The ratio corresponding to each data acquisition moment in a data collection cycle is obtained in advance
To data, later, the comparison data corresponding to the moment is acquired after the currently acquisition moment collects data, then from each data
Middle determination data corresponding with the current acquisition moment acquire the moment, later, determine that the data acquisition moment corresponding compares logarithm
According to.This is just needed after getting the comparison data in a data collection cycle corresponding to each data acquisition moment,
The corresponding data of each comparison data are acquired into moment corresponding label, i.e. comparison data information, comparison data information
In not only include comparison data, further include the acquisition moment corresponding with the comparison data.
Can also be：It only obtains data corresponding with the current acquisition moment and acquires moment corresponding comparison data.Working as
After the preceding acquisition moment collects data, then determine that one or more data corresponding with the current acquisition moment are adopted from historical data
Collect moment corresponding data, however, it is determined that be multiple data, then in different data collection cycles with the current acquisition moment pair
This multiple data is done average value, generates comparison data later by the data for the data acquisition moment acquisition answered.
Determination unit 43 is used to determine data wave rotating ring ratio according to the data and comparison data of the data source to be monitored of acquisition；
Determine that data wave rotating ring ratio is current in order to determine according to the data of the data source to be monitored of acquisition and comparison data
In data collection cycle, whether the fluctuation range between the data and comparison data of current time acquisition exceeds first threshold, when
When it exceeds first threshold, shows that the data are abnormal data, warning information is generated, to notify operating personnel to number to be monitored
It is handled according to source；When it is without departing from first threshold, show the data compared with comparison data, fluctuating range is in the first threshold
In value, belong to normal fluctuation range, which is that normal data is normally believed without sending out any information alternatively, only sending out data
Breath.
Further, while memory or database purchase have collected data, memory or database may be used also
To be stored with the data information for sending out warning information, such as：Some data acquisition moment in some data collection cycle adopts
The data collected have been issued warning information, then the data be abnormal data, using the data in memory or database as
When historical data participates in comparing, the abnormal data is excluded, avoids it from participating in comparing as comparison data, this reduces history
Data are as data error caused by comparison data.
Further, can also be：After having handled the data at current time, wherein handle the data at current time
May include：Abnormal data stores after sending out warning information, alternatively, normal data does not send out warning information, is only stored.
Later, from searching in the Current data acquisition period whether also have other untreated complete data in database, i.e., in current data
In collection period whether it is in need send out warning information, but since warning information caused by equipment shutdown or other reasons is not sent out
The case where, if so, handling the warning information, the warning information is sent out, also, need to include data in the warning information
Acquire the moment.
Specifically, can be：Make the difference the data of acquisition and comparison data to obtain the first difference, by the first difference with compare
Data are quotient, obtain data wave rotating ring ratio.
Specifically, if Current data acquisition moment collected data are A in the Current data acquisition period, history compares logarithm
Corresponding comparison data is B at the time of corresponding with the Current data acquisition moment in, then, data wave rotating ring ratio C is：
C=| A-BB |.
When C is more than first threshold, show the collected number of Current data acquisition moment institute in the Current data acquisition period
According to for abnormal data, needing to generate warning information.
Further, when C is less than first threshold, when detecting that last data acquires in the Current data acquisition period
It is abnormal data to carve the collected data of institute, and corresponding data wave rotating ring ratio is more than first threshold, due to Current data acquisition
The roll-collar ratio of moment collected data is less than first threshold, therefore, the alarm that the revocation last data acquisition moment generates
Information, to prompt staff's data to restore normal range (NR).
Generation unit 44 is used to, when data wave rotating ring ratio is more than first threshold, determine the number of the data source to be monitored of acquisition
According to for abnormal data, warning information is generated.
In data monitoring method disclosed in the present embodiment, can data acquisition be carried out to multiple equipment or system simultaneously,
Specifically, the monitoring to data can be realized by Browser/Server (Browser/Server Mode) program architecture,
In, Browser/Server can across multiple terminals displaying, such as：The ends PC, Phone, Pad etc. only need smart machine to provide browser,
Without providing client software packet to equipment various terminals, and it is possible to restful interfaces are provided based on http agreements, it is convenient
It is extended with other systems or integrated.
Data monitoring device disclosed in the present embodiment can also include：Predicting unit, wherein：
Sequencing at the time of predicting unit is used for according to acquisition comparison data, which generates, compares chart；According to the acquisition number
According at the time of generate data drawing list；Determine that Future Data acquires the moment according to the data variation trend for comparing chart and data drawing list
Prediction data trend.
Sequencing at the time of according to acquisition comparison data, which generates, compares chart, that is, obtains data collection cycle
Comparison data, the wherein comparison data of a data collection cycle can be the predetermined number in the Current data acquisition period
The average value of each data acquisition moment gathered data, the average value that each data are acquired to the moment form in data collection cycle
The comparison data of one complete data collection cycle.
The comparison data of one data collection cycle is acquired into the sequencing at moment according to corresponding data and generates ratio
To chart, specifically, the comparison chart can be a curve, in order to check the fluctuating range of its data.
Data drawing list is generated at the time of according to gathered data, can be specially：Within the Current data acquisition period, Mei Daoyi
A data acquire the moment with regard to gathered data, and the data are acquired moment and its corresponding data composition Current data acquisition period
A part in corresponding chart.
Specifically, can be as shown in Figure 5：
Darker curve is different data acquisition moment corresponding data in the Current data acquisition period, and lighter curve is advance
The comparison data of the complete data collection cycle obtained.
By the above chart it was determined that different data acquire the moment, the data in the Current data acquisition period with compare
The fluctuation size between the data in curve corresponding to data, and can be acquired according to multiple data before current in the moment
Data collection cycle moves towards to determine in the Current data acquisition period Current data acquisition moment with the curve in comparison data curve
The prediction data at one or several data acquisition moment in future later moves towards trend, and prediction future time instance data are extracted to realize
Trend.
Further, can also include：Data wave is determined according to the data of the data source to be monitored of acquisition and comparison data
Rotating ring ratio determines that the data of the data source to be monitored of acquisition are abnormal data when data wave rotating ring ratio is more than first threshold, raw
At warning information.
Data monitoring device disclosed in the present embodiment, by acquiring the data information of data source to be monitored, data packets
At the time of including the data and gathered data of acquisition, the comparison data of data source to be monitored is obtained, comparison data is gathered data
The historical data at moment determines data wave rotating ring ratio according to the data of acquisition and comparison data, when data wave rotating ring ratio is more than the
When one threshold value, determines that the data of the data source to be monitored of acquisition are abnormal data, generate warning information.This programme will be by directly will
The data of acquisition are compared with comparison data, when data wave rotating ring ratio is more than first threshold, that is, generate warning information,
In, it is handled without the data to acquisition, comparison process is simple to operation, without complicated data analysis process, simplifies
The algorithm of data monitoring, has saved data processing amount, improves the fluency of system operation.
Each embodiment is described by the way of progressive in this specification, the highlights of each of the examples are with other
The difference of embodiment, just to refer each other for identical similar portion between each embodiment.For device disclosed in embodiment
For, since it is corresponded to the methods disclosed in the examples, so description is fairly simple, related place is said referring to method part
It is bright.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, depends on the specific application and design constraint of technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest range caused.
Claims (10)
1. a kind of data monitoring method, which is characterized in that including：
The data information of data source to be monitored is acquired, the data information includes：The data of the data source to be monitored of acquisition
And at the time of acquiring the data；
Obtain the comparison data of the data source to be monitored, the history at the time of comparison data is the acquisition data
Data；
Data wave rotating ring ratio is determined according to the data of the data source to be monitored of the acquisition and the comparison data；
When the data wave rotating ring ratio is more than first threshold, determine that the data of the data source to be monitored of acquisition are abnormal number
According to generation warning information.
2. according to the method described in claim 1, it is characterized in that, the comparison data for obtaining the data source to be monitored,
The historical data at the time of comparison data is the acquisition data, including：
The comparison data information of the data source to be monitored is obtained, the comparison data information includes：The data source to be monitored
Comparison data and acquire the comparison data at the time of；
The acquisition comparison data corresponding at the time of the acquisition data is determined from the comparison data information
Moment was the first moment, obtained the comparison data of the first moment acquisition.
3. according to the method described in claim 2, it is characterized in that, further including：
Sequencing at the time of according to the acquisition comparison data, which generates, compares chart；
Data drawing list is generated at the time of according to the acquisition data；
Determine that Future Data acquires the prediction number at moment according to the data variation trend of the comparison chart and the data drawing list
According to trend.
4. according to the method described in claim 1, it is characterized in that, the comparison data for obtaining the data source to be monitored,
The historical data at the time of comparison data is the acquisition data, including：
Determine the data collection cycle of the data source to be monitored；
Obtain the historical data information of the predetermined period number acquired before the Current data acquisition period, the predetermined period number
Historical data information in include abnormal data information；
It determines the average value of the historical data of the predetermined period number, obtains comparison data information；
The acquisition comparison data corresponding at the time of the acquisition data is determined from the comparison data information
Moment was the first moment, obtained the comparison data of the first moment acquisition.
5. according to the method described in claim 1, it is characterized in that, the data source to be monitored according to the acquisition
Data and the comparison data determine data wave rotating ring ratio, including：
The data of the acquisition and the comparison data are made the difference to obtain the first difference, first difference is compared into logarithm with described
According to quotient is, data wave rotating ring ratio is obtained.
6. according to the method described in claim 1, it is characterized in that, further including：
If in the same data collection cycle, the data wave rotating ring ratio at current time is less than first threshold, and, previous moment
Data wave rotating ring ratio be more than first threshold, cancel the warning information.
7. a kind of data monitoring device, which is characterized in that including：Collecting unit, acquiring unit, determination unit and generation unit,
Wherein：
The collecting unit is used to acquire the data information of data source to be monitored, and the data information includes：The described of acquisition is waited for
At the time of the data and the acquisition data in monitoring data source；
The acquiring unit is used to obtain the comparison data of the data source to be monitored, and the comparison data is described in the acquisition
Historical data at the time of data；
The determination unit is used to determine number according to the data and the comparison data of the data source to be monitored of the acquisition
According to roll-collar ratio；
The generation unit is used to, when the data wave rotating ring ratio is more than first threshold, determine the data to be monitored of acquisition
The data in source are abnormal data, generate warning information.
8. device according to claim 7, which is characterized in that the acquiring unit is used for：
The comparison data information of the data source to be monitored is obtained, the comparison data information includes：The data source to be monitored
Comparison data and acquire the comparison data at the time of；It is determined from the comparison data information and the acquisition data
At the time of the corresponding acquisition comparison data at the time of be the first moment, obtain the comparison data of first moment acquisition.
9. device according to claim 8, which is characterized in that further include：Predicting unit, wherein：
The sequencing at the time of predicting unit is used for according to the acquisition comparison data, which generates, compares chart；Foundation
Data drawing list is generated at the time of the acquisition data；Become according to the data variation of the comparison chart and the data drawing list
Gesture determines the prediction data trend at Future Data acquisition moment.
10. device according to claim 7, which is characterized in that the acquiring unit is used for：
Determine the data collection cycle of the data source to be monitored；Obtain the predetermined period acquired before the Current data acquisition period
The historical data information of number does not include abnormal data information in the historical data information of the predetermined period number；Determine institute
The average value of the historical data of predetermined period number is stated, comparison data information is obtained；From the comparison data information determine with
It was the first moment at the time of the corresponding acquisition comparison data at the time of acquisition data, obtains first moment
The comparison data of acquisition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810353780.6A CN108667688A (en) | 2018-04-19 | 2018-04-19 | A kind of data monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810353780.6A CN108667688A (en) | 2018-04-19 | 2018-04-19 | A kind of data monitoring method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108667688A true CN108667688A (en) | 2018-10-16 |
Family
ID=63780220
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810353780.6A Pending CN108667688A (en) | 2018-04-19 | 2018-04-19 | A kind of data monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108667688A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110083131A (en) * | 2019-03-26 | 2019-08-02 | 石化盈科信息技术有限责任公司 | Technological parameter on-line early warning method and readable storage medium storing program for executing based on amplitude of variation |
| CN111176937A (en) * | 2019-12-19 | 2020-05-19 | 深圳猛犸电动科技有限公司 | Message middleware monitoring and warning system, method, terminal equipment and storage medium |
| CN111800297A (en) * | 2020-07-07 | 2020-10-20 | 浪潮云信息技术股份公司 | Snmp-based intelligent monitoring method and system for cloud physical host |
| CN112463541A (en) * | 2020-12-14 | 2021-03-09 | 上海金仕达软件科技有限公司 | Data monitoring method and system |
| CN113359670A (en) * | 2021-06-15 | 2021-09-07 | 宁波静合科技有限公司 | Marine diesel engine fault diagnosis method based on data driving |
| CN114817820A (en) * | 2022-06-30 | 2022-07-29 | 深圳希施玛数据科技有限公司 | Early warning method for website data upgrading and related device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150135312A1 (en) * | 2012-08-31 | 2015-05-14 | Hitachi, Ltd. | Service performance monitoring method |
| CN106789251A (en) * | 2016-12-23 | 2017-05-31 | 中国银行股份有限公司 | Net silver running state monitoring system and method |
| CN107705149A (en) * | 2017-09-22 | 2018-02-16 | 平安科技（深圳）有限公司 | Data method for real-time monitoring, device, terminal device and storage medium |
| CN107871190A (en) * | 2016-09-23 | 2018-04-03 | 阿里巴巴集团控股有限公司 | A kind of operational indicator monitoring method and device |
-
2018
- 2018-04-19 CN CN201810353780.6A patent/CN108667688A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150135312A1 (en) * | 2012-08-31 | 2015-05-14 | Hitachi, Ltd. | Service performance monitoring method |
| CN107871190A (en) * | 2016-09-23 | 2018-04-03 | 阿里巴巴集团控股有限公司 | A kind of operational indicator monitoring method and device |
| CN106789251A (en) * | 2016-12-23 | 2017-05-31 | 中国银行股份有限公司 | Net silver running state monitoring system and method |
| CN107705149A (en) * | 2017-09-22 | 2018-02-16 | 平安科技（深圳）有限公司 | Data method for real-time monitoring, device, terminal device and storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110083131A (en) * | 2019-03-26 | 2019-08-02 | 石化盈科信息技术有限责任公司 | Technological parameter on-line early warning method and readable storage medium storing program for executing based on amplitude of variation |
| CN111176937A (en) * | 2019-12-19 | 2020-05-19 | 深圳猛犸电动科技有限公司 | Message middleware monitoring and warning system, method, terminal equipment and storage medium |
| CN111800297A (en) * | 2020-07-07 | 2020-10-20 | 浪潮云信息技术股份公司 | Snmp-based intelligent monitoring method and system for cloud physical host |
| CN112463541A (en) * | 2020-12-14 | 2021-03-09 | 上海金仕达软件科技有限公司 | Data monitoring method and system |
| CN113359670A (en) * | 2021-06-15 | 2021-09-07 | 宁波静合科技有限公司 | Marine diesel engine fault diagnosis method based on data driving |
| CN114817820A (en) * | 2022-06-30 | 2022-07-29 | 深圳希施玛数据科技有限公司 | Early warning method for website data upgrading and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108667688A (en) | A kind of data monitoring method and device | |
| CN110135856B (en) | Repeated transaction risk monitoring method, device and computer readable storage medium | |
| US8938533B1 (en) | Automatic capture of diagnostic data based on transaction behavior learning | |
| CN105357038B (en) | Monitor the method and system of cluster virtual machine | |
| US11055169B2 (en) | Forecasting workload transaction response time | |
| CN106998317B (en) | Abnormal access requests recognition methods and device | |
| JP4570527B2 (en) | System performance monitoring program and system performance monitoring method | |
| CN110995781B (en) | Praise information processing method, device and system | |
| WO2018174262A1 (en) | Monitoring system, processing device, and monitoring device | |
| CN105453042A (en) | Pre-fetching content for service-connected applications | |
| CN103201724A (en) | Providing application high availability in highly-available virtual machine environments | |
| US11679792B2 (en) | Distributed bogie diagnostics for track monitoring | |
| CN111026621B (en) | Monitoring alarm method, device, equipment and medium for Elasticissearch cluster | |
| RU2012144650A (en) | AUTOMATED RECOVERY AND TRANSFER BY INSTANCE IN COMPLEX DISTRIBUTED APPLICATIONS | |
| CN103348346A (en) | Techniques for detecting new browser windows | |
| CN110704964A (en) | Steam turbine operation state diagnosis method and device, electronic equipment and storage medium | |
| JP6665503B2 (en) | Data collection system, data collection device and data collection method | |
| KR20200125115A (en) | Method and apparatus for improving security of data communiucation of unmanned warship using public network | |
| JP2018101201A (en) | Policy introduction effect prediction device, policy introduction effect prediction program and policy introduction effect prediction method | |
| CN110745163A (en) | Method, system and equipment for realizing rotating speed tracking sampling | |
| CN112148566B (en) | Method and device for monitoring computing engine, electronic equipment and storage medium | |
| CN103984857A (en) | System and method for monitoring Parkinson's disease | |
| CN116627771B (en) | Log acquisition method, device, electronic equipment and readable storage medium | |
| WO2020163230A1 (en) | Systems and methods for item response modelling of digital assessments | |
| CN110007940B (en) | Gray scale release verification method, system, server and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181016 |