CN109947713A - A kind of monitoring method and device of log - Google Patents
A kind of monitoring method and device of log Download PDFInfo
- Publication number
- CN109947713A CN109947713A CN201711052827.7A CN201711052827A CN109947713A CN 109947713 A CN109947713 A CN 109947713A CN 201711052827 A CN201711052827 A CN 201711052827A CN 109947713 A CN109947713 A CN 109947713A
- Authority
- CN
- China
- Prior art keywords
- keyword
- monitored
- frequency values
- frequency
- log data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of monitoring method of log and devices, are related to technical field of data processing, are to obtain from number of ways to solve the existing daily record data used, so as to cause daily record data the lower problem of accuracy.The method comprise the steps that obtaining log data；Determine keyword to be monitored；Count the keyword to be monitored corresponding frequency values in the log data；Judge the keyword to be monitored with the presence or absence of abnormal according to the frequency values；If so, outputting alarm information.The present invention is suitably applied in the monitoring to daily record data.
Description
Technical field
The present invention relates to technical field of data processing more particularly to the monitoring methods and device of a kind of log.
Background technique
With the continuous development of network technology, application of the network in life is also more and more extensive.Under normal conditions, network
A large amount of daily record data can be generated in operational process, and by can detecte abnormal data to the monitoring of log, and according to different
Regular data, which is searched, speculates failure that may be present, thereby, it is ensured that the accuracy and validity of daily record data are most important.
However, being in the prior art usually to obtain daily record data through a variety of ways, due to the data for acquiring log
Collection terminal is more, and the version of collection terminal is also very much, and these collected data are directly summarised in data processing end and are carried out
Processing uses, therefore not can guarantee the accuracy of the daily record data used.
Summary of the invention
In view of the above problems, the present invention provides the monitoring method and device of a kind of log, and main purpose is by original
Keyword to be monitored is monitored in beginning daily record data, to improve the accuracy of daily record data.
In order to solve the above technical problems, in a first aspect, the present invention provides a kind of monitoring method of log, this method packet
It includes:
Obtain log data；
Determine keyword to be monitored；
Count the keyword to be monitored corresponding frequency values in the log data；
Judge the keyword to be monitored with the presence or absence of abnormal according to the frequency values；
If so, outputting alarm information.
Optionally, the method also includes:
Obtain a plurality of history log data；
Multiple keywords are extracted from the history log data, and count each keyword history described in each item
Corresponding frequency values in daily record data；
When to be not less than first default for corresponding frequency values in the history log data described in each item for the keyword
Frequency threshold and be no more than the second predeterminated frequency threshold value when, extract the keyword, obtain effective keyword.
Optionally, the method also includes:
Calculate the average frequency value of each effective keyword；
According to the frequency values in each effective keyword history log data described in each item, extract with it is described average
The maximum maximum fluctuation frequency values of frequency value difference；
Using the difference between the maximum fluctuation frequency values and the average frequency value divided by the average frequency value, obtain
To fluctuation range corresponding with each effective keyword；
According to the average frequency value and the fluctuation range, frequency corresponding with each effective keyword is calculated
Rate value range.
Optionally, described to judge that the keyword to be monitored includes: with the presence or absence of abnormal according to the frequency values
Extract effective keyword corresponding with the keyword to be monitored；
The frequency values of the keyword to be monitored are detected whether within the scope of the frequency values of effective keyword；
If so, judging the keyword to be monitored, there is no abnormal；
Otherwise, judge described to be monitored crucial in the presence of abnormal.
Optionally, the method also includes:
Effective keyword is updated according to preset time interval.
Second aspect, the present invention also provides a kind of monitoring device of log, which includes:
Acquiring unit, for obtaining log data；
Determination unit, for determining keyword to be monitored；
Statistic unit, for counting the keyword to be monitored corresponding frequency values in the log data；
Judging unit, for judging the keyword to be monitored with the presence or absence of abnormal according to the frequency values；
Output unit, for if so, outputting alarm information.
Optionally, described device further include: extraction unit,
The acquiring unit is also used to obtain a plurality of history log data；
The extraction unit is also used to extract multiple keywords from the history log data；
The statistic unit, is also used to count and respectively corresponds in each keyword history log data described in each item
Frequency values；
The extraction unit is also used to when the keyword corresponding frequency in the history log data described in each item
Value extracts the keyword not less than the first predeterminated frequency threshold value and when being no more than the second predeterminated frequency threshold value, obtains effectively
Keyword.
Optionally, described device further include: computing unit,
The computing unit, for calculating the average frequency value of each effective keyword；
The extraction unit is also used to according to the frequency in each effective keyword history log data described in each item
Rate value, extraction differ maximum maximum fluctuation frequency values with the average frequency value；
The computing unit is also used to remove using the difference between the maximum fluctuation frequency values and the average frequency value
With the average frequency value, fluctuation range corresponding with each effective keyword is obtained；
The computing unit, is also used to according to the average frequency value and the fluctuation range, and calculating each has with described
Imitate the corresponding frequency values range of keyword.
Optionally, the judging unit includes:
Extraction module, for extracting effective keyword corresponding with the keyword to be monitored；
Detection module, for detect the keyword to be monitored frequency values whether effective keyword frequency values
In range；
First judgment module, if for the keyword to be monitored frequency values effective keyword frequency values model
In enclosing, then judging the keyword to be monitored, there is no abnormal；
Second judgment module, if the frequency values for the keyword to be monitored are not in the frequency values of effective keyword
In range, then it is abnormal to judge that the keyword to be monitored exists.
Optionally, described device further include:
Updating unit, for being updated according to preset time interval to effective keyword.
To achieve the goals above, according to the third aspect of the invention we, a kind of storage medium, the storage medium are provided
Program including storage, wherein equipment where controlling the storage medium in described program operation executes day described above
The monitoring method of will.
To achieve the goals above, according to the fourth aspect of the invention, a kind of processor is provided, the processor is used for
Run program, wherein described program executes the monitoring method of log described above when running.
By above-mentioned technical proposal, the monitoring method and device of log provided by the invention, for using in the prior art
Daily record data obtain through a variety of ways, since data collection terminal quantity and version are all more complicated and not to log number
According to being monitored, so that the accuracy of data is unable to get guarantee, the present invention, which passes through, obtains log data, and according to
After different demands determines keyword to be monitored, each keyword that statistics needs to monitor is corresponding in log data
Frequency values, and judge that keyword to be monitored exists with the presence or absence of exception, and in keyword frequency values according to obtained statistical result
Warning message is exported in the case where exception, therefore compared with the prior art, the present invention can be in log data to key
Word frequency rate carries out statistics and judge keyword with the presence or absence of abnormal, in the daily record data that ensures to use later according to statistical result
There is no abnormal datas, to improve the accuracy of daily record data；In addition, being carried out according to the method for the present invention to daily record data
Monitoring, by whether Yi Chang effective keyword is updated in time for measuring keyword, it can be ensured that the timeliness of data
Property, so as to further improve the accuracy of daily record data.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 shows a kind of monitoring method flow chart of log provided in an embodiment of the present invention；
Fig. 2 shows the monitoring method flow charts of another log provided in an embodiment of the present invention；
Fig. 3 shows a kind of composition block diagram of the monitoring device of log provided in an embodiment of the present invention；
Fig. 4 shows the composition block diagram of the monitoring device of another log provided in an embodiment of the present invention.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
It is fully disclosed to those skilled in the art.
In order to improve the accuracy of daily record data, the embodiment of the invention provides a kind of monitoring methods of log, such as Fig. 1 institute
Show, this method comprises:
101, log data is obtained.
Wherein, the log data is the daily record data without any treatment measures.Specifically, clothes can be passed through
Business device obtains, embodiments, the embodiment of the present invention such as access, crawler capturing are not specifically limited in this embodiment from the background.
It should be noted that can be configuration in the server for log for the executing subject of the embodiment of the present invention
The device being monitored illustrates to need the keyword in daily record data this moment when device has detected daily record data input
It is monitored, triggers acquisition instruction, and then realize the monitoring to keyword in log data.
102, keyword to be monitored is determined.
Wherein, the keyword to be monitored can be an individual word, or the combination of multiple words, such as to
Monitoring keyword can be " Beijing TV ", or " Hunan Satellite TV, happy base camp " does not do this embodiment of the present invention
It is specific to limit.
Specifically, the word that user is needed to be monitored by the user demand that the step 102 can be different according to is true
It is set to keyword to be monitored, or according to some representative keywords determined without use occasion, the present invention
Embodiment is not specifically limited in this embodiment.
103, keyword to be monitored corresponding frequency values in the log data are counted.
Wherein, the frequency values can be any numbers such as 3,59,100.Specifically, this step can pass through statistical function
It is counted, can also be grabbed by crawler and calculates the corresponding frequency values of each keyword, the embodiment of the present invention is not done
It is specific to limit.
Further, it for the ease of data processing and use, for the embodiment of the present invention, can also be wrapped after this step
Include will obtained corresponding frequency values of each keyword of statistics and keyword is corresponding is stored in corresponding tables of data, but
It is without being limited thereto.Such as in log data, statistics obtains keyword 1, keyword to be monitored 2 and key to be monitored to be monitored
The corresponding frequency values of word 3 are 22,18 and 74, then can be stored in statistical result in tables of data as shown in Table 1:
Table 1
By the way that the obtained frequency values of statistics and keyword are stored in tables of data accordingly, can make need using
Lookup use can be carried out when keyword frequency values directly in tables of data, avoids search in a large amount of unordered data making
At the problem of wasting time, to improve the efficiency of data processing, further improve the efficiency of log monitoring.
104, judge the keyword to be monitored with the presence or absence of abnormal according to the frequency values.
Wherein, there is abnormal can uprush for keyword frequency values or anticlimax etc. in the keyword to be monitored.Specifically,
This step can use comparison function and be compared the frequency values of keyword to be monitored and preset frequency values, but unlimited
In this.
If 105, there is abnormal, outputting alarm information in the keyword to be monitored.
Wherein, the warning information can be text warning information, picture warning information, phonic warning information etc., this hair
Bright embodiment is not specifically limited in this embodiment.
The monitoring method of log provided in an embodiment of the present invention is by more for daily record data used in the prior art
What kind of approach obtained, since data collection terminal quantity and version are all more complicated and be not monitored to daily record data, to make
The accuracys of data is unable to get guarantees, the present invention by obtaining log data, and according to different needs determination to
After monitoring keyword, each keyword that statistics needs monitor corresponding frequency values in log data, and according to
It is abnormal that the statistical result arrived judges that keyword to be monitored whether there is, and deposits in keyword frequency values and export in an exceptional case
Warning message, therefore compared with the prior art, the present invention can count simultaneously keyword frequency in log data
According to statistical result judge keyword with the presence or absence of abnormal, in the daily record data that uses after ensuring there is no abnormal data,
To improve the accuracy of daily record data；In addition, be monitored according to the method for the present invention to daily record data, by being used for
Measure keyword whether Yi Chang effective keyword is updated in time, it can be ensured that the timeliness of data, so as to more into
The accuracy of the raising daily record data of one step.
Further, as the refinement and extension to embodiment illustrated in fig. 1, the embodiment of the invention also provides another days
The monitoring method of will, as shown in Figure 2.
201, a plurality of history log data is obtained.
Wherein, a plurality of history log data can be the history log data in preset historical time section, and every
History log data can be to be divided with preset time period, and the daily record data in each time cycle is a data,
But not limited to this.For example, obtaining the history log data in the past in 1 month, and divided according to the time cycle " day ", it will be every
It daily record data is as a history log data.
202, multiple keywords are extracted from the history log data, and count each keyword described in each item
Corresponding frequency values in history log data.
Wherein, the representative keyword for the application scenarios selection that the keyword can be different according to, can also
Think according to Predistribution Algorithm carry out it is random extract keyword, such as keyword news, amusement, black can be extracted for playing log
Longjiang satellite TV etc., and as described in above-mentioned steps 102, the keyword of extraction can be a word, or multiple words
The combination of group, the embodiment of the present invention are not specifically limited in this embodiment.
In addition, frequency values of each keyword described in this step in each daily record data can be 0,14,288 etc.
Random natural number value shows in this daily record data not when frequency values of some keyword in certain entry will data are 0
There are the keywords.And specifically, it can be inquired in each daily record data by query function and count each key
The frequency values of word, can also be in such a way that crawler grabs in daily record data, and the embodiment of the present invention is not specifically limited in this embodiment.
203, when the keyword, corresponding frequency values are not less than first in the history log data described in each item
Predeterminated frequency threshold value and be no more than the second predeterminated frequency threshold value when, extract the keyword.
Further, effective keyword is obtained.Wherein, the first predeterminated frequency threshold value and the second predeterminated frequency threshold value can
To be set according to the actual conditions of different keywords, the embodiment of the present invention is not specifically limited.Pass through extraction in this step
The existing and keyword of the frequency value difference in every daily record data in a certain range, real in each daily record data
Now to the screening of keyword, and this class keywords filtered out are determined as effective keyword.
For example, 7 history in the week age of getting over play daily record data, in this 7 history natural number logs
Keyword variety, news and amusement are extracted in data and counts the corresponding frequency values of these three keywords, are obtained such as the following table 2
Shown in result:
Table 2
Wherein, keyword " variety " does not occur in the 2nd article of broadcasting history log data, and keyword " news " exists
The frequency phase-difference occurred in this 7 daily record datas is larger, so being to the result of these three keywords screening, by keyword " joy
It is happy " it is determined as effective keyword.
Further, after the step 203 the method also includes: calculate each effective keyword and be averaged
Frequency values；According to the frequency values in each effective keyword history log data described in each item, extract with it is described average
The maximum maximum fluctuation frequency values of frequency value difference；Utilize the difference between the maximum fluctuation frequency values and the average frequency value
Value obtains fluctuation range corresponding with each effective keyword divided by the average frequency value；According to the average frequency
Value and the fluctuation range calculate frequency values range corresponding with each effective keyword.
Such as above-mentioned effective keyword " amusement ", it is flat that it is calculated according to frequency of the keyword in 7 history log datas
Equal frequency values are (56+50+50+54+50+55+56)/7=53, and extraction differs maximum maximum fluctuation frequency with the average frequency value
Rate value is 50 or 56, so fluctuation range (56-53) 53=is calculated according to maximum fluctuation frequency values and average frequency value
5.7%, so effectively keyword " amusement " corresponding frequency values range 53 ± 53*5.7%=50~56.It should be noted that
In practical application scene, appropriate adjustment can be carried out to the fluctuation range being calculated according to the actual situation, in order to improve
The accuracy rate or fault-tolerant ability of daily record data.
For the embodiment of the present invention, pass through the frequency of each keyword in a plurality of daily record data in the statistical history period
Value, and after screening obtains effective keyword, the average frequency value and fluctuation range of each keyword are calculated, and then be calculated
The frequency values range of each effective keyword, in order to can be directly compared when judging whether keyword to be monitored is abnormal
Judgement, improves the efficiency of keyword judgement to be monitored, and then improves the efficiency of log monitoring.
204, log data is obtained.
Wherein, log data concept explanation described in this step and specific embodiment refer to the step
It is accordingly described in 101, details are not described herein.
205, keyword to be monitored is determined.
Wherein, the concept explanation of the keyword to be monitored can be referred to and accordingly be described in the step 102, herein no longer
It repeats.
It should be noted that for the embodiment of the present invention, the step of above-mentioned acquisition log data and in this step really
The sequencing of fixed keyword to be monitored can be interchanged, and also may be performed simultaneously.
206, keyword to be monitored corresponding frequency values in the log data are counted.
Specifically, the corresponding concept explanation of this step and embodiment are identical with the step 103, no longer superfluous herein
It states.
207, effective keyword corresponding with the keyword to be monitored is extracted.
Wherein, effective keyword corresponding with keyword to be monitored can be identical with keyword to be monitored
Keyword, or include keyword combination of keyword to be monitored etc..Specifically, can be carried out according to keyword identification
It extracts, inquiry extraction etc. can also be carried out according to query function.For the embodiment of the present invention, corresponding to each effective keyword
Frequency values range computation after the completion of, each effective keyword and the corresponding frequency values range of effective keyword can be stored in
In tables of data, thus when this step needs to extract with keyword to be monitored to effective keyword of drink, it can be directly in data
It carries out searching extraction and use in table, for example, preserving whole using regular expression according to the corresponding character string of keyword
Lookup extraction is carried out in the tables of data of the frequency range of effective keyword and effective keyword, is obtained corresponding with keyword to be monitored
Effective keyword.By the way that directly inquiry uses data directly in the tables of data for preserving effective keyword, avoid big
It measures and carries out the problem of wasting time caused by lookup extraction in unordered data, to improve the efficiency of data processing, in turn
Improve the efficiency of log monitoring.
208, the frequency values of the keyword to be monitored are detected whether within the scope of the frequency values of effective keyword.
For the embodiment of the present invention, since the frequency range of effective keyword is counted according to a large amount of history log datas
It arrives, thus by the way that the frequency values of keyword to be monitored are compared with the frequency range of effective keyword, it can be accurate
Whether the frequency of occurrences for measuring keyword to be monitored belongs to normal condition, supervises so as to realize to log data
Control.
209a, if so, judging the keyword to be monitored, there is no abnormal.
For the embodiment of the present invention, when there are multiple keywords to be monitored, if judging, a keyword to be monitored is not deposited
In exception, then monitoring of the triggering to next keyword to be monitored at this time, until all keywords to be monitored have all judged
Finish.
The step 209b arranged side by side, otherwise with the step 209a, judge it is described it is to be monitored it is crucial exist it is abnormal.
As described in above-mentioned steps, when the frequency model for effective keyword that the frequency values of keyword to be monitored are not obtained in statistics
When enclosing interior, show that the situation does not meet frequency values rule under normal conditions, so that the keyword to be monitored is determined as exception
Keyword.
For the embodiment of the present invention, by the frequency values of key to be monitored in statistics log data, and by the frequency
Value is compared with the frequency range for screening obtained effective keyword in advance, thus the frequency values of keyword to be monitored whether
Judge that keyword with the presence or absence of exception, can accurately be monitored initial data, in order to find in the frequency range
It is found in time when keyword exception, it is ensured that the accuracy of the subsequent daily record data used.
Step 210b is executed after the step 209b, if so, outputting alarm information.
Wherein, the concept explanation of the warning information can be no longer superfluous herein with reference to the corresponding description in the step 105
It states.
In order to further increase the accuracy of log monitoring, the method can also include: according to preset time interval pair
Effective keyword is updated.Wherein, the preset time interval can for a week, one month or 6 months etc.,
The embodiment of the present invention is not specifically limited in this embodiment.For the embodiment of the present invention, by whether abnormal for measuring keyword
Effective keyword is updated in time, it can be ensured that the timeliness of data, so as to further improve daily record data
Accuracy.
Further, the method according to step 201-210b, the embodiment of the present invention can also provide one and combine specifically
Application scenarios under monitor log embodiment, the implementation process is divided into five execution steps, specifically, as described below:
The first step sets daily daily record data as a daily record data, and one month in the past is obtained from database
30 interior history log datas, the keyword extracted are ten keywords such as keyword 1, keyword 2 ... keyword 10,
Statistics this corresponding frequency values of 10 keywords in 30 daily record datas.
Second step passes through the existing and corresponding frequency in each daily record data in every daily record data of screening conditions
Whether the keyword that rate value is not much different screens 10 keywords, finally obtain different for measuring keyword to be monitored
Normal effective keyword is 9 three keyword 2, keyword 5 and keyword keywords.
Third step, successively according to frequency values of these three effective keywords in 30 daily record datas calculate these three effectively
The average frequency value of keyword, and extract each effective keyword and differ maximum frequency values calculating fluctuation model with average frequency value
It encloses, the frequency range of each effective keyword is determined according further to average frequency value and fluctuation range.
4th step obtains log data, and it is right in the log data to count determining keyword A to be monitored
The frequency values answered are 12, are inquired and are extracted and key to be monitored in the tables of data for preserving effective keyword using query function
The corresponding effective keyword 2 of word A.
5th step, according to the corresponding frequency range of effective keyword 2 recorded in tables of data, 32~34, due to be monitored
The frequency values 12 of keyword A are not in the frequency range, so it is abnormal to judge that keyword A exists, output includes " keyword A
The text warning information of data exception ".
But it should be recognized that specific embodiment described in above-mentioned application scenarios is only exemplary, not this hair
Unique specific embodiment of bright embodiment is only one of the optimal enforcement mode of method of the present invention that meets herein.
Further, as the realization to method shown in above-mentioned Fig. 1, the embodiment of the invention also provides a kind of prisons of log
Device is controlled, for realizing to above-mentioned method shown in FIG. 1.The Installation practice is corresponding with preceding method embodiment, for just
In reading, present apparatus embodiment no longer repeats the detail content in preceding method embodiment one by one, it should be understood that this
Device in embodiment can correspond to the full content realized in preceding method embodiment.As shown in figure 3, the device includes: to obtain
Unit 31, determination unit 32, statistic unit 33, judging unit 34, output unit 35 are taken, wherein
Acquiring unit 31 can be used for obtaining log data.
Determination unit 32 is determined for keyword to be monitored.
Statistic unit 33 can be used for counting the determining keyword to be monitored of the determination unit 32 in the acquiring unit
Corresponding frequency values in 31 log datas got；
Judging unit 34 can be used for judging the pass to be monitored according to the frequency values that the statistic unit 33 statistics obtains
Keyword is with the presence or absence of abnormal；
Output unit 35, it is defeated if can be used for the judging unit 34 judges that the keyword to be monitored has exception
Warning information out.
Further, as the realization to method shown in above-mentioned Fig. 2, the embodiment of the invention also provides another logs
Monitoring device, for being realized to above-mentioned method shown in Fig. 2.The Installation practice is corresponding with preceding method embodiment, is
Easy to read, present apparatus embodiment no longer repeats the detail content in preceding method embodiment one by one, it should be understood that
Device in the present embodiment can correspond to the full content realized in preceding method embodiment.As shown in figure 4, the device includes:
Acquiring unit 41, determination unit 42, statistic unit 43, judging unit 44, output unit 45, wherein
Acquiring unit 41 can be used for obtaining log data.
Determination unit 42 is determined for keyword to be monitored.
Statistic unit 43 can be used for counting the determining keyword to be monitored of the determination unit 42 in the acquiring unit
Corresponding frequency values in 41 log datas got；
Judging unit 44 can be used for judging the pass to be monitored according to the frequency values that the statistic unit 43 statistics obtains
Keyword is with the presence or absence of abnormal；
Output unit 45, it is defeated if can be used for the judging unit 44 judges that the keyword to be monitored has exception
Warning information out.
Further, described device further include: extraction unit 46.
The acquiring unit 41 can be also used for obtaining a plurality of history log data.
The extraction unit 46 can be used for from the history log data extracting multiple keywords.
The statistic unit 43 can be also used for counting in each keyword history log data described in each item and divide
Not corresponding frequency values.
Further, described device further include: computing unit 47.
The extraction unit 46 can be also used for corresponding in the history log data described in each item when the keyword
Frequency values be not less than the first predeterminated frequency threshold value and be no more than the second predeterminated frequency threshold value when, extract the keyword.
The computing unit 47 can be used for calculating the average frequency value of each effective keyword.
The extraction unit 46 can be also used for according to each effective keyword history log data described in each item
In frequency values, extraction maximum maximum fluctuation frequency values are differed with the average frequency value.
The computing unit 47 can be also used for using between the maximum fluctuation frequency values and the average frequency value
Difference obtains fluctuation range corresponding with each effective keyword divided by the average frequency value.
Computing unit 47, can be also used for according to the average frequency value and the fluctuation range, calculate with it is described each
The corresponding frequency values range of effective keyword.
Further, described device further include: detection unit 48, judging unit 49
The extraction unit 46 can be also used for extracting effective keyword corresponding with the keyword to be monitored.
The detection unit 48, whether the frequency values that can be also used for detecting the keyword to be monitored are in effective pass
Within the scope of the frequency values of keyword.
The judging unit 49, if can be used for frequency of the frequency values in effective keyword of the keyword to be monitored
Within the scope of rate value, then judging the keyword to be monitored, there is no abnormal.
The judging unit 49 can be also used for otherwise, judging described to be monitored crucial in the presence of abnormal.
The monitoring device of another kind log provided in an embodiment of the present invention.Described device includes: acquiring unit, determines list
Member, statistic unit, judging unit and output unit.It is to obtain through a variety of ways for daily record data used in the prior art
, since data collection terminal quantity and version are all more complicated and be not monitored to daily record data, so that the standard of data
True property is unable to get guarantee, and the present invention determines keyword to be monitored by obtaining log data according to different needs
Later, statistics needs each keyword monitored corresponding frequency values in log data, and according to obtained statistics knot
It is abnormal that fruit judges that keyword to be monitored whether there is, and deposits in keyword frequency values and export warning message in an exceptional case,
Therefore compared with the prior art, the present invention can count keyword frequency in log data and be tied according to statistics
Fruit judges keyword with the presence or absence of abnormal, with there is no abnormal datas in the daily record data that uses after ensuring, to improve
The accuracy of daily record data；In addition, being monitored according to the method for the present invention to daily record data, by for measuring keyword
Whether Yi Chang effective keyword is updated in time, it can be ensured that the timeliness of data, so as to further improve
The accuracy of daily record data.
The text processing apparatus includes processor and memory, above-mentioned acquiring unit 41, determination unit 42, statistic unit
43, judging unit 44, output unit 45 etc. store in memory as program unit, are stored in storage by processor execution
Above procedure unit in device realizes corresponding function.
Include kernel in processor, is gone in memory to transfer corresponding program unit by kernel.Kernel can be set one
Or more, the accuracy of daily record data monitoring is improved by adjusting kernel parameter.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, if read-only memory (ROM) or flash memory (flash RAM), memory include that at least one is deposited
Store up chip.
The embodiment of the invention provides a kind of storage mediums, are stored thereon with program, real when which is executed by processor
The monitoring method of the existing log.
The embodiment of the invention provides a kind of processor, the processor is for running program, wherein described program operation
The monitoring method of log described in Shi Zhihang.
The embodiment of the invention provides a kind of equipment, equipment include processor, memory and storage on a memory and can
The program run on a processor, processor perform the steps of acquisition log data when executing program；It determines to be monitored
Keyword；Count the keyword to be monitored corresponding frequency values in the log data；Sentenced according to the frequency values
The keyword to be monitored that breaks is with the presence or absence of abnormal；If so, outputting alarm information.
Further, the method also includes:
Obtain a plurality of history log data；
Multiple keywords are extracted from the history log data, and count each keyword history described in each item
Corresponding frequency values in daily record data；
When to be not less than first default for corresponding frequency values in the history log data described in each item for the keyword
Frequency threshold and be no more than the second predeterminated frequency threshold value when, extract the keyword, obtain effective keyword.
Further, the method also includes:
Calculate the average frequency value of each effective keyword；
According to the frequency values in each effective keyword history log data described in each item, extract with it is described average
The maximum maximum fluctuation frequency values of frequency value difference；
Using the difference between the maximum fluctuation frequency values and the average frequency value divided by the average frequency value, obtain
To fluctuation range corresponding with each effective keyword；
According to the average frequency value and the fluctuation range, frequency corresponding with each effective keyword is calculated
Rate value range.
Further, described to judge that the keyword to be monitored includes: with the presence or absence of abnormal according to the frequency values
Extract effective keyword corresponding with the keyword to be monitored；
The frequency values of the keyword to be monitored are detected whether within the scope of the frequency values of effective keyword；
If so, judging the keyword to be monitored, there is no abnormal；
Otherwise, judge described to be monitored crucial in the presence of abnormal.
Further, the method also includes:
Effective keyword is updated according to preset time interval.
Equipment in the embodiment of the present invention can be server, PC, PAD, mobile phone etc..
The embodiment of the invention also provides a kind of computer program products, when executing on data processing equipment, are suitable for
It executes the program of initialization there are as below methods step: obtaining log data；Determine keyword to be monitored；Statistics is described wait supervise
Control keyword corresponding frequency values in the log data；Judge that the keyword to be monitored is according to the frequency values
It is no to there is exception；If so, outputting alarm information.
Further, the method also includes:
Obtain a plurality of history log data；
Multiple keywords are extracted from the history log data, and count each keyword history described in each item
Corresponding frequency values in daily record data；
When to be not less than first default for corresponding frequency values in the history log data described in each item for the keyword
Frequency threshold and be no more than the second predeterminated frequency threshold value when, extract the keyword, obtain effective keyword.
Further, the method also includes:
Calculate the average frequency value of each effective keyword；
According to the frequency values in each effective keyword history log data described in each item, extract with it is described average
The maximum maximum fluctuation frequency values of frequency value difference；
Using the difference between the maximum fluctuation frequency values and the average frequency value divided by the average frequency value, obtain
To fluctuation range corresponding with each effective keyword；
According to the average frequency value and the fluctuation range, frequency corresponding with each effective keyword is calculated
Rate value range.
Further, described to judge that the keyword to be monitored includes: with the presence or absence of abnormal according to the frequency values
Extract effective keyword corresponding with the keyword to be monitored；
The frequency values of the keyword to be monitored are detected whether within the scope of the frequency values of effective keyword；
If so, judging the keyword to be monitored, there is no abnormal；
Otherwise, judge described to be monitored crucial in the presence of abnormal.
Further, the method also includes:
Effective keyword is updated according to preset time interval.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable Jie
The example of matter.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory
(DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electrically erasable
Read memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM), the more function of number
Can CD (DVD) other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices or it is any its
His non-transmission medium, can be used for storing and can be accessed by a computing device information.As defined in this article, computer-readable Jie
Matter does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence " including one ... ", it is not excluded that including element
Process, method, there is also other identical elements in commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The above is only embodiments herein, are not intended to limit this application.To those skilled in the art,
Various changes and changes are possible in this application.It is all within the spirit and principles of the present application made by any modification, equivalent replacement,
Improve etc., it should be included within the scope of the claims of this application.
Claims (10)
1. a kind of monitoring method of log, which is characterized in that the described method includes:
Obtain log data；
Determine keyword to be monitored；
Count the keyword to be monitored corresponding frequency values in the log data；
Judge the keyword to be monitored with the presence or absence of abnormal according to the frequency values；
If so, outputting alarm information.
2. the method according to claim 1, wherein the method also includes:
Obtain a plurality of history log data；
Multiple keywords are extracted from the history log data, and count each keyword history log described in each item
Corresponding frequency values in data；
When the keyword in the history log data described in each item corresponding frequency values be not less than the first predeterminated frequency
Threshold value and be no more than the second predeterminated frequency threshold value when, extract the keyword, obtain effective keyword.
3. according to the method described in claim 2, it is characterized in that, the method also includes:
Calculate the average frequency value of each effective keyword；
According to the frequency values in each effective keyword history log data described in each item, extract and the average frequency
Value differs maximum maximum fluctuation frequency values；
Using the difference between the maximum fluctuation frequency values and the average frequency value divided by the average frequency value, obtain with
The corresponding fluctuation range of each effective keyword；
According to the average frequency value and the fluctuation range, frequency values corresponding with each effective keyword are calculated
Range.
4. according to the method described in claim 3, it is characterized in that, described judge the key to be monitored according to the frequency values
Word whether there is exception
Extract effective keyword corresponding with the keyword to be monitored；
The frequency values of the keyword to be monitored are detected whether within the scope of the frequency values of effective keyword；
If so, judging the keyword to be monitored, there is no abnormal；
Otherwise, judge described to be monitored crucial in the presence of abnormal.
5. the method according to any one of claim 2-4, which is characterized in that the method also includes:
Effective keyword is updated according to preset time interval.
6. a kind of monitoring device of log, which is characterized in that described device includes:
Acquiring unit, for obtaining log data；
Determination unit, for determining keyword to be monitored；
Statistic unit, for counting the keyword to be monitored corresponding frequency values in the log data；
Judging unit, for judging the keyword to be monitored with the presence or absence of abnormal according to the frequency values；
Output unit, for if so, outputting alarm information.
7. device according to claim 6, which is characterized in that described device further include: extraction unit,
The acquiring unit is also used to obtain a plurality of history log data；
The extraction unit is also used to extract multiple keywords from the history log data；
The statistic unit is also used to count corresponding frequency in each keyword history log data described in each item
Rate value；
The extraction unit, be also used to when the keyword in the history log data described in each item corresponding frequency values it is equal
When not less than the first predeterminated frequency threshold value and being no more than the second predeterminated frequency threshold value, the keyword is extracted, effective key is obtained
Word.
8. device according to claim 7, which is characterized in that described device further include: computing unit,
The computing unit, for calculating the average frequency value of each effective keyword；
The extraction unit is also used to according to the frequency in each effective keyword history log data described in each item
Value, extraction differ maximum maximum fluctuation frequency values with the average frequency value；
The computing unit is also used to using the difference between the maximum fluctuation frequency values and the average frequency value divided by institute
Average frequency value is stated, fluctuation range corresponding with each effective keyword is obtained；
The computing unit is also used to be calculated and each effective pass according to the average frequency value and the fluctuation range
The corresponding frequency values range of keyword.
9. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein run in described program
When control the storage medium where equipment perform claim require 1 to the log described in any one of claim 5 monitoring
Method.
10. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run
Benefit require 1 to the log described in any one of claim 5 monitoring method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711052827.7A CN109947713B (en) | 2017-10-31 | 2017-10-31 | Log monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711052827.7A CN109947713B (en) | 2017-10-31 | 2017-10-31 | Log monitoring method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109947713A true CN109947713A (en) | 2019-06-28 |
| CN109947713B CN109947713B (en) | 2021-08-10 |
Family
ID=67003954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711052827.7A Active CN109947713B (en) | 2017-10-31 | 2017-10-31 | Log monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109947713B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110690992A (en) * | 2019-09-16 | 2020-01-14 | 中盈优创资讯科技有限公司 | Network cutover abnormity identification method and device |
| CN113406935A (en) * | 2021-06-22 | 2021-09-17 | 惠民万顺节能新材料有限公司 | Monitoring system for hot-dip aluminum zinc plate production process |
| CN113761133A (en) * | 2021-09-10 | 2021-12-07 | 未鲲(上海)科技服务有限公司 | System abnormity monitoring method and device based on artificial intelligence and related equipment |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102740247A (en) * | 2011-04-15 | 2012-10-17 | 中国移动通信集团山东有限公司 | Method and device for generating warning message |
| CN103164424A (en) * | 2011-12-13 | 2013-06-19 | 阿里巴巴集团控股有限公司 | Method and device for acquiring time-efficient words |
| CN103401710A (en) * | 2013-07-30 | 2013-11-20 | 浙江中烟工业有限责任公司 | Variance-based firewall abnormal log detection method |
| CN104036034A (en) * | 2014-06-30 | 2014-09-10 | 百度在线网络技术（北京）有限公司 | Log analysis method and device for data warehouse |
| CN104899127A (en) * | 2014-03-04 | 2015-09-09 | 腾讯数码（天津）有限公司 | Monitoring method and device of server |
| CN105183912A (en) * | 2015-10-12 | 2015-12-23 | 北京百度网讯科技有限公司 | Abnormal log determination method and device |
| WO2016131383A1 (en) * | 2015-07-16 | 2016-08-25 | 中兴通讯股份有限公司 | Method and device for running maintenance process of system |
| CN106202511A (en) * | 2016-07-21 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | A kind of alarm method based on log analysis and system |
| CN106991145A (en) * | 2017-03-23 | 2017-07-28 | 中国银联股份有限公司 | A kind of method and device of Monitoring Data |
-
2017
- 2017-10-31 CN CN201711052827.7A patent/CN109947713B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102740247A (en) * | 2011-04-15 | 2012-10-17 | 中国移动通信集团山东有限公司 | Method and device for generating warning message |
| CN103164424A (en) * | 2011-12-13 | 2013-06-19 | 阿里巴巴集团控股有限公司 | Method and device for acquiring time-efficient words |
| CN103401710A (en) * | 2013-07-30 | 2013-11-20 | 浙江中烟工业有限责任公司 | Variance-based firewall abnormal log detection method |
| CN104899127A (en) * | 2014-03-04 | 2015-09-09 | 腾讯数码（天津）有限公司 | Monitoring method and device of server |
| CN104036034A (en) * | 2014-06-30 | 2014-09-10 | 百度在线网络技术（北京）有限公司 | Log analysis method and device for data warehouse |
| WO2016131383A1 (en) * | 2015-07-16 | 2016-08-25 | 中兴通讯股份有限公司 | Method and device for running maintenance process of system |
| CN105183912A (en) * | 2015-10-12 | 2015-12-23 | 北京百度网讯科技有限公司 | Abnormal log determination method and device |
| CN106202511A (en) * | 2016-07-21 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | A kind of alarm method based on log analysis and system |
| CN106991145A (en) * | 2017-03-23 | 2017-07-28 | 中国银联股份有限公司 | A kind of method and device of Monitoring Data |
Non-Patent Citations (1)
| Title |
|---|
| 岳阳化工总厂科技处: "《全面质量管理基础知识与统计方法》", 31 January 1980 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110690992A (en) * | 2019-09-16 | 2020-01-14 | 中盈优创资讯科技有限公司 | Network cutover abnormity identification method and device |
| CN110690992B (en) * | 2019-09-16 | 2022-03-29 | 中盈优创资讯科技有限公司 | Network cutover abnormity identification method and device |
| CN113406935A (en) * | 2021-06-22 | 2021-09-17 | 惠民万顺节能新材料有限公司 | Monitoring system for hot-dip aluminum zinc plate production process |
| CN113761133A (en) * | 2021-09-10 | 2021-12-07 | 未鲲(上海)科技服务有限公司 | System abnormity monitoring method and device based on artificial intelligence and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109947713B (en) | 2021-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105550184B (en) | A kind of information acquisition method and device | |
| CN103685914B (en) | Image management apparatus and management method | |
| US20140201129A1 (en) | Efficient query processing using histograms in a columnar database | |
| CN106656536A (en) | Method and device for processing service invocation information | |
| CN108829715A (en) | For detecting the method, equipment and computer readable storage medium of abnormal data | |
| CN109947713A (en) | A kind of monitoring method and device of log | |
| CN106649316A (en) | Video pushing method and device | |
| CN110020339B (en) | Webpage data acquisition method and device based on non-buried point | |
| CN105930363A (en) | HTML5 webpage based user behavior analysis method and device | |
| CN107578263A (en) | A kind of detection method, device and the electronic equipment of advertisement abnormal access | |
| CN108023764A (en) | Abnormality eliminating method and device | |
| CN109561052A (en) | The detection method and device of website abnormal flow | |
| US11423009B2 (en) | System and method to prevent formation of dark data | |
| CN108255886B (en) | Evaluation method and device of recommendation system | |
| CN105653949B (en) | A kind of malware detection methods and device | |
| CN109828993A (en) | A kind of querying method and device of statistical data | |
| CN108810268A (en) | Processing method and device for operation record | |
| CN106937173B (en) | Video playing method and device | |
| CN110399405A (en) | Log alarming method, apparatus, system and storage medium | |
| CN110490639A (en) | A kind of data volume monitoring method, modification method, system and computer equipment | |
| CN110020074A (en) | Determine the method and device of webpage turnover rate | |
| CN107392220A (en) | The clustering method and device of data flow | |
| CN105573999B (en) | The method and apparatus of identification intelligent equipment user | |
| CN106789392A (en) | A kind of methods, devices and systems for monitoring web crawlers | |
| CN110991241A (en) | Abnormality recognition method, apparatus, and computer-readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |