CN105049260B - Blog management method and device - Google Patents
Blog management method and device Download PDFInfo
- Publication number
- CN105049260B CN105049260B CN201510523986.5A CN201510523986A CN105049260B CN 105049260 B CN105049260 B CN 105049260B CN 201510523986 A CN201510523986 A CN 201510523986A CN 105049260 B CN105049260 B CN 105049260B
- Authority
- CN
- China
- Prior art keywords
- log
- buffer queue
- new
- existing
- queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007726 management method Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 17
- 230000008569 process Effects 0.000 claims description 10
- 241000239290 Araneae Species 0.000 claims 1
- 238000012216 screening Methods 0.000 abstract description 7
- 238000012423 maintenance Methods 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
Abstract
The present invention provides a kind of blog management method and devices.It is related to cluster management field；Solve the problems, such as that log quantity excessively influences system performance.This method comprises: one log buffer queue of maintenance, stores generated log in the log buffer queue；When generating new log, check in the log buffer queue with the presence or absence of existing log identical with the new log；When there is existing log identical with the new log, the new log being incorporated into the existing log, does not add the new log individually corresponding entry in the log buffer queue.Technical solution provided by the invention is suitable for cluster management system, realizes the screening management of log information.
Description
Technical field
The present invention relates to cluster management field more particularly to a kind of blog management methods and device.
Background technique
In cluster management system, system need each resource in cycle detection cluster state (such as IP resource, using journey
Sequence resource, Service Source etc.), while the result that can be will test is recorded in journal file.Since detection process is uninterruptedly to follow
What ring carried out, therefore will cause ceaselessly record log information, causing log information to be postponed at any time can become very huge, very
To influence system performance；On the other hand, if deleting these logs, though can be reduced log quantity, it will lead to the later period once going out
When existing problem, can not effective position problem analysis, the effect of log information cannot be played.
Summary of the invention
The present invention provides a kind of blog management method and devices, and solving log quantity excessively influences asking for system performance
Topic.
The present invention provides a kind of blog management methods, comprising:
It safeguards a log buffer queue, stores generated log in the log buffer queue；
When generating new log, check in the log buffer queue with the presence or absence of existing day identical with the new log
Will；
When there is existing log identical with the new log, the new log is incorporated into the existing log, no
The new log individually corresponding entry is added in the log buffer queue.
Preferably, the new log is incorporated into the existing log specifically:
The log generation time of the existing log is updated to the generation time of the new log.
Preferably, it when generating new log, checks in the log buffer queue with the presence or absence of identical as the new log
Existing log the step of after, further includes:
When there is no existing log identical with the new log, the new log is added to the log buffer team
In column.
Preferably, it checks specific with the presence or absence of existing log identical with the new log in the log buffer queue
Are as follows:
It checks in the log added in threshold time in the log buffer queue with the presence or absence of identical as the new log
Existing log.
Preferably, the log buffer queue is the entry that every log distributes one, and an entry includes following content
In it is any or any number of:
Log generation time, log content, log add up frequency of occurrence.
Preferably, this method further include:
The log in the log buffer queue is arranged according to log generation time.
Preferably, this method further include:
After the generation time for having updated the existing log, this is had into log and is inserted into queue heads position, and should
The log of existing log adds up frequency of occurrence and increases by 1.
The present invention also provides a kind of log management apparatus, comprising:
Queue memory module stores generated day in the log buffer queue for safeguarding a log buffer queue
Will；
Retrieval module whether there is and the new day for when generating new log, checking in the log buffer queue
The identical existing log of will；
Log update module will the new log merging for when there is existing log identical with the new log
To the existing log, the new log individually corresponding entry is not added in the log buffer queue.
Preferably, the log update module is specifically used for:
The log generation time of the existing log is updated to the generation time of the new log.
Preferably, the log update module is also used to when there is no existing log identical with the new log, will
The new log is added in the log buffer queue.
The present invention provides a kind of blog management method and devices, a log buffer queue are safeguarded, in the log buffer team
Generated log is stored in column, when generating new log, checking in the log buffer queue whether there is and the new day
The identical existing log of will, when there is existing log identical with the new log, by the new log be incorporated into it is described
There is log, does not add the new log individually corresponding entry in the log buffer queue.Realize the sieve of log information
Reason is checked, solves the problems, such as that log quantity excessively influences system performance.
Detailed description of the invention
Fig. 1 is a kind of flow chart for blog management method that the embodiment of the present invention 1 provides；
Fig. 2 is the structural schematic diagram of log information buffer queue；
Fig. 3 is the process flow diagram in the absence of aiming in buffer queue new day；
Fig. 4 is process flow diagram when new day aiming at existing in buffer queue；
Fig. 5 is a kind of structural schematic diagram for log management apparatus that the embodiment of the present invention two provides.
Specific embodiment
Since detection process is that uninterrupted circulation carries out, it will cause ceaselessly record log information, lead to log
Information is postponed at any time can become very huge, or even influence system performance；On the other hand, if deleting these logs, though energy
Reduce log quantity, but will lead to the later period once when something goes wrong, can not effective position problem analysis, log information cannot be played
Effect.
In view of above-mentioned analysis, the log management module for designing a high efficiency smart will be very necessary, it may ensure that in energy
In the case where enough effective record log information, repetition and the redundant logs information without reference to break-up value are filtered out automatically, greatly
Reduce the difficulty of influence and log analysis of the huge log amount to system.
To solve the above-mentioned problems, the embodiment provides a kind of blog management method and devices.It below will knot
Accompanying drawings and embodiments are closed technical solution of the present invention is described in detail.The log information that system generates will not direct nothing
Differentially be recorded in journal file, but analysis filtering carried out by the log module first, will wherein excessively repeat and nothing
The log of break-up value filters out, and only remains with the log of reference value, for example, if a certain item record resource status
Log repeats in a short time, then the practical log for only first appearing and finally occurring needs to record, what centre occurred
Log only illustrates that the state is cycle detection, not too big break-up value, therefore can ignore.
It should be noted that each feature in the embodiment of the present invention and embodiment can be tied mutually if do not conflicted
It closes, it is within the scope of the present invention.In addition, though logical order is shown in flow charts, but in certain situations
Under, it can be with the steps shown or described are performed in an order that is different from the one herein.
First in conjunction with attached drawing, the embodiment of the present invention one is illustrated.
The embodiment of the invention provides a kind of blog management methods, and process such as Fig. 1 of log management is completed using this method
It is shown, comprising:
Step 101, one log buffer queue of maintenance, store generated log in the log buffer queue；
The log buffer queue is the entry that every log distributes one, and an entry includes any in following content
Or it is any number of:
Log generation time, log content, log add up frequency of occurrence.
As shown in Fig. 2, creation log information buffer queue will save in journal queue for temporarily storing log information
The number that creation time, the content of log and the log of log information occur, new log will be inserted into queue head.
Step 102, when generating new log, check in the log buffer queue with the presence or absence of identical as the new log
Existing log；
Step 103, when there is no existing log identical with the new log, the new log is added to the day
In will buffer queue；
In this step, specifically, when the log generation time of the existing log is updated to the generation of the new log
Between.
As shown in figure 3, according to the content of log A, inquiry should from queue after system generates a new log information A
Whether log has existed, if it find that the log is not present in the queue, then log is inserted into queue head, wherein Atime
For the time for generating the log, AtextFor the particular content of log, AnumFor the log occur number, due to be for the first time go out
It is existing, therefore AnumValue be 1.On the other hand, the log occurred for the first time will be recorded directly in journal file.
Step 104, when there is existing log identical with the new log, the new log is incorporated into described existing
The new log individually corresponding entry is not added in log in the log buffer queue；
After the generation time for having updated the existing log, this is had into log and is inserted into queue heads position, and should
The log of existing log adds up frequency of occurrence and increases by 1.
As shown in figure 4, when newly generated log B has existed in the queue, then by elder generation the log B in queue from team
It is deleted in column, the generation time of B is then updated to the newest time by the original time, i.e., by Bold_tIt is updated to Bnew_t, together
When by frequency of occurrence BnumAfter adding 1, it is reinserted into the head of queue.It by this process, may be implemented, the day repeated
Will is always in the position close to queue head, and unduplicated log will be gradually moved into queue end as time goes by
Position.
In addition, the log in the log buffer queue can be also arranged in the embodiment of the present invention according to log generation time.
In addition, a settable threshold time, checking in the log added in threshold time in the log buffer queue is
It is no to there is existing log identical with the new log.Specifically, checking log in queue whether by the way of traversal
More than threshold time T, i.e., whether the difference of the generation time and current time of log is more than T, since unduplicated log is concentrated
In queue end position, and the log of repeatability is intended in queue head, therefore when traversing since queue ending, when discovery
Between be more than T log, then it is recorded in journal file in chronological order, while being removed from queue, and in ergodic process
In, once finding that a certain logging time is less than T, then terminate epicycle traversal.
With reference to the accompanying drawing, the embodiment of the present invention two is illustrated.
The embodiment of the invention provides a kind of log management apparatus, structure is as shown in Figure 5, comprising:
Queue memory module 501 stores generated for safeguarding a log buffer queue in the log buffer queue
Log；
Retrieval module 502, for when generating new log, check in the log buffer queue with the presence or absence of with it is described newly
The identical existing log of log；
Log update module 503 will the new log conjunction for when there is existing log identical with the new log
And to the existing log, the new log individually corresponding entry is not added in the log buffer queue.
Preferably, the log update module 503 is specifically used for:
The log generation time of the existing log is updated to the generation time of the new log.
Preferably, the log update module 503 is also used to when there is no existing logs identical with the new log
When, the new log is added in the log buffer queue.
In the embodiment of the present invention, following mechanism is defined to realize above-mentioned function:
1) log information buffer queue mechanism
Buffer queue is generated for interim storage system, carries out the log of analysis screening.Buffer queue includes day
The content of time, log that will generates and log add up three contents of frequency of occurrence.The time that buffer queue is generated according to log
It is ranked up, newest log is inserted into the head position of queue on the time.
2) log content duplicate checking is analyzed
After receiving a log information, by inquiry log buffer queue, judge whether the log has existed.If
Log is not present, then illustrates that the log is the log occurred for the first time within this time, then the log is inserted into caching team
In column, and the time t of log generation and the number (being at this time 1) of log appearance are recorded in the queue, while the log
Information will also directly write in journal file；If illustrating this by finding that the log is already present in queue after inquiry
Log is to repeat log, then after deleting the same log in queue, which is inserted into queue heads position, while by log
Frequency of occurrence adds 1, and the generation time of this log also will be updated to the most newly generated time.
By this process, repetition aims to guarantee only one in buffer queue day, and it is forward to substantially remain in queue
Position, and the log not repeated in queue is then gradually discharged to the position at the close end of queue.
3) overtime rule mechanism
For the log information that system generates, after filtering screening, day will be finally recorded in satisfactory log
It is saved in will file.Therefore for the log information in queue, log module will be checked slow by sequence from the front to the back
Deposit whether the time that the log in queue generates has been more than the threshold time T of module setting, once find that there are the generations of log
Time is more than T, then illustrates that the log does not repeat in a short time, needs to delete the log information from queue at this time
Remove, and according to log generation time by the log recording into journal file.
The embodiment of the present invention provide technical solution be equally applicable to other can recycle printing repeat log system or
Software can effectively reduce log quantity by carrying out automatic screening to log information, improve the efficiency of log analysis and accurate
Property.In addition, after can also slightly improving, leading to if needing to know to repeat the information such as the time interval of log when carrying out log analysis
The technical solution of the embodiment of the present invention offer is crossed to realize.
The embodiment provides a kind of blog management method and devices, a log buffer queue are safeguarded, in the day
Generated log is stored in will buffer queue, when generating new log, check in the log buffer queue with the presence or absence of with
The identical existing log of the new log will the new log merging when there is existing log identical with the new log
To the existing log, the new log individually corresponding entry is not added in the log buffer queue.Realize log
The screening management of information solves the problems, such as that log quantity excessively influences system performance.
The technical solution that the embodiment of the present invention provides can be realized centralized management and screening to log information, it is ensured that
In the case where not omitting important log, the quantity of repeatability and the log without break-up value can be reduced as far as possible；Due to can be real
Now to the automatic screening of log information, therefore when carrying out log design layout, it may not be necessary to which whether take notice of very much can occurrence log
Situations such as circulation generates improves the development efficiency of system or software；Other than clustered software, if other systems or software are deposited
It can recycle and lead to the problem of in certain logs, the technical solution that the embodiment of the present invention offer also can be used solves.
Those of ordinary skill in the art will appreciate that all or part of the steps in the above method can be instructed by program
Related hardware is completed, and described program can store in computer readable storage medium, such as read-only memory, disk or CD
Deng.Optionally, one or more integrated circuits can be used also to realize in all or part of the steps of above-described embodiment.Accordingly
Ground, each module/unit in above-described embodiment can take the form of hardware realization, can also use the shape of software function module
Formula is realized.The present invention is not limited to the combinations of the hardware and software of any particular form.
Certainly, the invention may also have other embodiments, without deviating from the spirit and substance of the present invention, ripe
It knows those skilled in the art and makes various corresponding changes and modifications, but these corresponding changes and change in accordance with the present invention
Shape all should belong to scope of protection of the claims of the invention.
Claims (4)
1. a kind of blog management method characterized by comprising
It safeguards a log buffer queue, stores generated log in the log buffer queue；When generating new log, check
With the presence or absence of existing log identical with the new log in the log buffer queue；
When there is existing log identical with the new log, first the existing log is deleted from the log buffer queue
It removes, then the log generation time of the existing log is updated to the generation time of the new log, and this is had into log
It is reinserted into queue heads position；
By traversal mode from the front to the back, the difference of the log generation time and current time in the log buffer queue is checked
Whether value is more than threshold time；By be more than the threshold time log recording into journal file, and by it from the log
It is removed in buffer queue；And in ergodic process, if traversing a certain log generation time less than the threshold time, terminate
Epicycle traversal；
When generating new log, check in the log buffer queue with the presence or absence of existing log identical with the new log
After step, further includes:
When there is no existing log identical with the new log, the new log is inserted into queue heads position, and should
Log is recorded directly in the journal file.
2. blog management method according to claim 1, which is characterized in that the log buffer queue is every log point
With one entry, an entry includes any or any number of in following content:
Log generation time, log content, log add up frequency of occurrence.
3. blog management method according to claim 2, which is characterized in that this method further include:
After the generation time for having updated the existing log, which is added up into frequency of occurrence and increases by 1.
4. a kind of log management apparatus characterized by comprising
Queue memory module stores generated log in the log buffer queue for safeguarding a log buffer queue；
Retrieval module whether there is and the new log phase for when generating new log, checking in the log buffer queue
Same existing log；
Log update module first deletes the existing log from the log buffer queue, then by the existing log
Log generation time be updated to the generation time of the new log, and this is had into log and is reinserted into queue heads position；
Spider module, for checking the log generation time in the log buffer queue by traversal mode from the front to the back
Whether the difference with current time is more than threshold time；By the log recording more than the threshold time into journal file, and
It is removed from the log buffer queue；And in ergodic process, if traversing a certain log generation time less than described
Threshold time then terminates epicycle traversal；
The log update module is also used to when there is no existing log identical with the new log, by the new log
It is inserted into queue heads position, and the log is recorded directly in the journal file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510523986.5A CN105049260B (en) | 2015-08-24 | 2015-08-24 | Blog management method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510523986.5A CN105049260B (en) | 2015-08-24 | 2015-08-24 | Blog management method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105049260A CN105049260A (en) | 2015-11-11 |
| CN105049260B true CN105049260B (en) | 2019-03-19 |
Family
ID=54455482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510523986.5A Active CN105049260B (en) | 2015-08-24 | 2015-08-24 | Blog management method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105049260B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105515855B (en) * | 2015-12-04 | 2018-10-30 | 浪潮(北京)电子信息产业有限公司 | The daily record playback optimization method and device deleted directly is entangled in a kind of cloud storage system |
| CN107870849B (en) * | 2016-09-28 | 2021-03-09 | 平安科技（深圳）有限公司 | Processing method and device of test log |
| JP6474772B2 (en) * | 2016-09-28 | 2019-02-27 | 本田技研工業株式会社 | Communication status judgment method |
| CN106649615B (en) * | 2016-11-30 | 2020-09-22 | 苏州浪潮智能科技有限公司 | Log segment management method and device |
| CN106777049B (en) * | 2016-12-09 | 2021-01-01 | 武汉斗鱼网络科技有限公司 | Processing method and system for avoiding repeated log output |
| CN106844143A (en) * | 2016-12-27 | 2017-06-13 | 微梦创科网络科技（中国）有限公司 | A kind of daily record duplicate removal treatment method and device |
| CN108491316A (en) * | 2018-04-03 | 2018-09-04 | 郑州云海信息技术有限公司 | A kind of blog management method, device and electronic equipment |
| CN109617737A (en) * | 2018-12-27 | 2019-04-12 | 携程计算机技术（上海）有限公司 | The monitoring method and system of the log platform of internet |
| CN110191005B (en) * | 2019-06-25 | 2020-02-21 | 北京九章云极科技有限公司 | Alarm log processing method and system |
| CN117056188B (en) * | 2023-10-13 | 2024-01-30 | 西安热工研究院有限公司 | Method, system, equipment and storage medium for preventing log storm on DCS controller |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103164511A (en) * | 2013-02-21 | 2013-06-19 | 烽火通信科技股份有限公司 | Method for storage event log to automatically filter repeated jitter data |
| CN103761262A (en) * | 2013-12-31 | 2014-04-30 | 深圳英飞拓科技股份有限公司 | Repetition log control method based on syslogd |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201417445Y (en) * | 2009-04-15 | 2010-03-03 | 卡斯柯信号有限公司 | Dynamic multi-output log processing device |
| CN101729295B (en) * | 2009-12-02 | 2012-01-11 | 北京东土科技股份有限公司 | Method for realizing log function |
| CN101950293B (en) * | 2010-08-11 | 2012-11-21 | 东软集团股份有限公司 | Log extraction method and device |
-
2015
- 2015-08-24 CN CN201510523986.5A patent/CN105049260B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103164511A (en) * | 2013-02-21 | 2013-06-19 | 烽火通信科技股份有限公司 | Method for storage event log to automatically filter repeated jitter data |
| CN103761262A (en) * | 2013-12-31 | 2014-04-30 | 深圳英飞拓科技股份有限公司 | Repetition log control method based on syslogd |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105049260A (en) | 2015-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105049260B (en) | Blog management method and device | |
| US7827299B2 (en) | Transitioning between historical and real time data streams in the processing of data change messages | |
| US11249943B2 (en) | Scalable enterprise content management | |
| US20160321294A1 (en) | Distributed, Scalable Key-Value Store | |
| US9489282B2 (en) | Selective purging of a log structure | |
| CN108647151A (en) | It is a kind of to dodge system metadata rule method, apparatus, equipment and storage medium entirely | |
| CN107122360A (en) | Data mover system and method | |
| CN107122355A (en) | Data mover system and method | |
| CN101673192B (en) | Method for time-sequence data processing, device and system therefor | |
| US8972338B2 (en) | Sampling transactions from multi-level log file records | |
| CN103631940A (en) | Data writing method and data writing system applied to HBASE database | |
| DE112018003585B4 (en) | A method, computer program product, and storage tape drive hardware unit for enhancing deduplication of tape drive storage | |
| CN107122361A (en) | Data mover system and method | |
| US9031909B2 (en) | Provisioning and/or synchronizing using common metadata | |
| US10031948B1 (en) | Idempotence service | |
| CN109491589A (en) | A kind of delamination process and device based on file content | |
| US8838615B2 (en) | Computer implemented method for automatically managing stored checkpoint data | |
| CN110147203A (en) | A kind of file management method, device, electronic equipment and storage medium | |
| CN105302488B (en) | The method for writing data and system of a kind of storage system | |
| US9953295B2 (en) | Management of event contexts using bookend contexts | |
| CN102597995B (en) | Synchronizing database and non-database resources | |
| US10169357B2 (en) | Methods and systems for data cleanup using physical image of files on storage devices | |
| CN106815232A (en) | Catalog management method, apparatus and system | |
| US10423901B2 (en) | Management of event contexts using bookend events | |
| US20190303476A1 (en) | Dynamic buffer pools for process non-conforming tasks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |