CN103064779A - Method for processing log information rapidly - Google Patents
Method for processing log information rapidly Download PDFInfo
- Publication number
- CN103064779A CN103064779A CN201210575837XA CN201210575837A CN103064779A CN 103064779 A CN103064779 A CN 103064779A CN 201210575837X A CN201210575837X A CN 201210575837XA CN 201210575837 A CN201210575837 A CN 201210575837A CN 103064779 A CN103064779 A CN 103064779A
- Authority
- CN
- China
- Prior art keywords
- log
- log information
- buffer module
- log buffer
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a method for processing log information rapidly. The method includes step 1, setting three log caching modules for every log in a system memory and marking every log caching module with numbers, step 2, storing log information produced by the system in the log caching modules in turn and step 3, detecting states of every log caching module in real time and transmitting the log information in the log caching modules which are in full states to journal files. By the utilization of the method for processing log information rapidly, log information is input into the journal files of a disc after the log information is accumulated by caching the log information so that frequent input-output operations which are required once is greatly reduced and the processing performance of the system is greatly improved. Besides, the sizes of the log caching module, the sizes of the journal files and no-output time of logs and the like can be arranged according to different requirements.
Description
Technical field
The present invention relates to computer realm, relate in particular to a kind of method of fast processing log information.
Background technology
Business scale expansion along with intelligent switching platform, processing speed for service message requires also more and more higher, owing in the message of managing business, can produce a large amount of log informations, log information is input to frequently in the journal file of disk, can cause the decline of system's handling property.Therefore need to improve the treatment effeciency of log information, reduce the input-output operation of disk.
Summary of the invention
Technical matters to be solved by this invention is for the deficiencies in the prior art, but a kind of fast processing log information is provided, and greatly reduces the method for the fast processing log information of disk input-output operation.
The technical scheme that the present invention solves the problems of the technologies described above is as follows: a kind of method of fast processing log information, it is characterized in that, and comprise the steps:
Step 1: in Installed System Memory, for each daily record arranges three log buffer modules, and be each log buffer module label;
Step 2: the log information that system is produced stores in the log buffer module in turn;
Step 3: detect in real time the state of each log buffer module, the log information that will be in the log buffer module that expires state is transferred in the journal file.
On the basis of technique scheme, the present invention can also do following improvement.
Further, the state of described log buffer module comprise idle condition, with state with expired state.
Further, the concrete steps that the log information that in the described step 2 system is produced stores in the log buffer module in turn are as follows:
Step 2.1: detect in real time the state of each log buffer module, judge whether to exist the log buffer module that is in state, enter step 2.2 if exist, otherwise enter step 2.3;
Step 2.2: judge whether the log buffer module that is in state also has enough spatial caches, if also have enough spatial caches then log information is stored in this log store module, otherwise the status modifier of above-mentioned log buffer module is become to expire state;
Step 2.3: judge whether to exist the log buffer module that is in idle condition, enter step 2.4 if exist, otherwise prompting occurs unusually;
Step 2.4: judge the number of the log buffer module that is in idle condition, if only there is one, then log information is stored in this log buffer module; If two or more are arranged, then according to the label of log buffer module, in order log information is stored in the log buffer module.
Further, technique scheme comprises that also the output of the rank that presets log information and log information limits rank, and other log information of output restricted class that meets predefined log information that will be in the log buffer module that expires state outputs in the journal file.
Further, technique scheme also comprises when the log information of daily record memory module output exceeds the capacity of journal file, then automatically generates the fraternal file of journal file, log information is cut apart, and stored into respectively in former journal file and the fraternal file.
Further, technique scheme also comprises and also comprises after arriving the predetermined daily record no-output time still not having log information to be input in the journal file, then the full content in the log buffer district is outputed in the journal file automatically.
Further, technique scheme also comprises according to size, the size of journal file, the daily record no-output time of different demands to the log buffer module and arranging.
The invention has the beneficial effects as follows: adopt method provided by the invention, by log information is carried out buffer memory, be input in the journal file of disk after log information saved bit by bit, this greatly reduces the frequently input-output operation of original needs, has greatly improved the handling property of system again; The size, daily record no-output time of size, the journal file of log buffer module etc. can be set for different demands simultaneously.
Description of drawings
Fig. 1 is the method flow diagram of a kind of fast processing log information of the present invention;
Fig. 2 is the particular flow sheet of step 2 of the present invention;
Embodiment
Below in conjunction with accompanying drawing principle of the present invention and feature are described, institute gives an actual example and only is used for explaining the present invention, is not be used to limiting scope of the present invention.
As shown in Figure 1, a kind of method of fast processing log information is characterized in that, comprises the steps:
Step 1: in Installed System Memory, for each daily record arranges three log buffer modules, and be each log buffer module label;
Step 2: the log information that system is produced stores in the log buffer module in turn;
Step 3: detect in real time the state of each log buffer module, the log information that will be in the log buffer module that expires state is transferred in the journal file.
Wherein, the log buffer module is arranged in the internal memory, and journal file is arranged in the disk.
Wherein, the state of described log buffer module comprise idle condition, with state with expired state.
Wherein, as shown in Figure 2, the concrete steps that the log information that in the described step 2 system is produced stores in the log buffer module in turn are as follows:
Step 2.1: detect in real time the state of each log buffer module, judge whether to exist the log buffer module that is in state, enter step 2.2 if exist, otherwise enter step 2.3;
Step 2.2: judge whether the log buffer module that is in state also has enough spatial caches, if there are enough spatial caches, then log information is stored in this log store module, otherwise the status modifier of above-mentioned log buffer module is become to expire state;
Step 2.3: judge whether to exist the log buffer module that is in idle condition, enter step 2.4 if exist, otherwise prompting occurs unusually;
Step 2.4: judge the number of the log buffer module that is in idle condition, if only there is one, then log information is stored in this log buffer module; If two or more are arranged, then according to the label of log buffer module, in order log information is stored in the log buffer module.
Wherein, technique scheme comprises that also the output of the rank that presets log information and log information limits rank, other log information of output restricted class that meets predefined log information that will be in the log buffer module that expires state outputs in the journal file, such as, set log information and comprise 5 ranks, it is 3 grades that the output of the log information of setting limits rank, then when having the log buffer module to be in to expire state, front 3 grades log information is outputed in the journal file, and the 4th, 5 grade log information is then removed.
Wherein, technique scheme also comprises when the log information of daily record memory module output exceeds the capacity of journal file, then automatically generate the fraternal file of journal file, log information is cut apart, and store into respectively in former journal file and the fraternal file, in the time of can not holding the log information of all log buffer modules outputs such as the capacity of journal file A, then journal file A adds suffix automatically with the filename of self, become A.1, and automatically generate fraternal file A.2, log information is cut apart, then store into respectively journal file A.1 with fraternal file A.2 in.
Wherein, technique scheme also comprises after arriving the predetermined daily record no-output time does not still have log information to be input in the journal file, then automatically output to the full content in the log buffer module in the journal file, such as, preset a time T, when reaching time T, output in the journal file without any log information, then all log informations in the log buffer module are all outputed in the journal file.
Wherein, technique scheme arranged according to size, the size of journal file, the daily record no-output time of different demands to the log buffer module.
Below the present invention is described in detail:
1, open up three log buffer modules at first for each daily record and be used for storing daily record information, the state of each log buffer module idle, with completely circulating between three states.
2, after the first block buffer is full, state variation is for full, when the first block buffer is in completely in the state, uses the second block buffering module.
3, whether to have the state of log buffer module be to have expired state to Real-Time Monitoring, if exist then the content in the corresponding buffer module is outputed to journal file, thereby many required repeatedly input-output operations of log recording reduced to 1 time.
Even 4, three cache modules use in turn and can guarantee also can have when the daily record amount is very huge time enough to carry out input-output operation, log information is stored in the log buffer module, after having the log buffer module to be filled with, all log informations in this log buffer module are once outputed in the journal file.
Can size that judge journal file when 5, outputing to the log information in the log buffer module in the journal file hold all log informations to be entered, if could directly log information that all are to be entered be input in the journal file, if could not would automatically generate the fraternal file of former journal file, log information to be entered is cut apart, be input to respectively in former journal file and the fraternal journal file.
6, preset a time parameter, if after reaching the schedule time, there is not log information to output in the journal file, then automatically the log information in all log buffer modules is outputed in the journal file.
Adopt method provided by the invention, by log information is carried out buffer memory, be input in the journal file of disk after log information saved bit by bit again, this greatly reduces the frequently input-output operation of original needs, has greatly improved the handling property of system; The size, daily record no-output time of size, the journal file of log buffer module etc. can be set for different demands simultaneously.
The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of doing, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (7)
1. the method for a fast processing log information is characterized in that, comprises the steps:
Step 1: in Installed System Memory, for each daily record arranges three log buffer modules, and be each log buffer module label;
Step 2: the log information that system is produced stores in the log buffer module in turn;
Step 3: detect in real time the state of each log buffer module, the log information that will be in the log buffer module that expires state is transferred in the journal file.
2. the method for described a kind of quick reason speed log information according to claim 1 is characterized in that, the state of described log buffer module comprises idle condition, with state with expired state.
3. the method for described a kind of quick reason speed log information according to claim 1 and 2 is characterized in that, the concrete steps that the log information that in the described step 2 system is produced stores in the log buffer module in turn are as follows:
Step 2.1: detect in real time the state of each log buffer module, judge whether to exist the log buffer module that is in state, enter step 2.2 if exist, otherwise enter step 2.3;
Step 2.2: judge whether the log buffer module that is in state also has enough spatial caches, if enough spatial caches that also have then log information is stored in this log store module, otherwise the status modifier of above-mentioned log buffer module is become to expire state;
Step 2.3: judge whether to exist the log buffer module that is in idle condition, enter step 2.4 if exist, otherwise prompting occurs unusually;
Step 2.4: judge the number of the log buffer module that is in idle condition, if only there is one, then log information is stored in this log buffer module; If two or more are arranged, then according to the label of log buffer module, in order log information is stored in the log buffer module.
4. the method for described a kind of quick reason speed log information according to claim 1, it is characterized in that, the output that also comprises the rank that presets log information and log information limits rank, and other log information of output restricted class that meets predefined log information that will be in the log buffer module that expires state outputs in the journal file.
5. the method for described a kind of quick reason speed log information according to claim 1, it is characterized in that, also comprise when the log information of daily record memory module output exceeds the capacity of journal file, then automatically generate the fraternal file of journal file, log information is cut apart, and stored into respectively in former journal file and the fraternal file.
6. the method for described a kind of quick reason speed log information according to claim 1, it is characterized in that, also comprise after arriving the predetermined daily record no-output time still not having log information to be input in the journal file, then the full content in the log buffer district is outputed in the journal file automatically.
7. the method for described a kind of quick reason speed log information according to claim 1 is characterized in that, also comprises according to size, the size of journal file, the daily record no-output time of different demands to the log buffer module arranging.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210575837.XA CN103064779B (en) | 2012-12-26 | 2012-12-26 | A kind of method of quick process log information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210575837.XA CN103064779B (en) | 2012-12-26 | 2012-12-26 | A kind of method of quick process log information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103064779A true CN103064779A (en) | 2013-04-24 |
| CN103064779B CN103064779B (en) | 2016-08-03 |
Family
ID=48107412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210575837.XA Active CN103064779B (en) | 2012-12-26 | 2012-12-26 | A kind of method of quick process log information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103064779B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793517A (en) * | 2014-02-12 | 2014-05-14 | 浪潮电子信息产业股份有限公司 | File system log dump dynamic capacity-increase method based on monitoring mechanism |
| CN105760283A (en) * | 2014-12-18 | 2016-07-13 | 阿里巴巴集团控股有限公司 | Log output method and device |
| CN106951488A (en) * | 2017-03-14 | 2017-07-14 | 海尔优家智能科技（北京）有限公司 | A kind of log recording method and device |
| CN108804295A (en) * | 2017-04-28 | 2018-11-13 | 北京京东尚科信息技术有限公司 | log information recording method and device |
| CN111723130A (en) * | 2020-06-28 | 2020-09-29 | 杭州安恒信息技术股份有限公司 | Historical data query method, device and equipment |
| CN112527612A (en) * | 2020-11-06 | 2021-03-19 | 北京罗克维尔斯科技有限公司 | System log preservation method and device, vehicle-mounted system and vehicle |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1932812A (en) * | 2005-09-16 | 2007-03-21 | 腾讯科技（深圳）有限公司 | Method and apparatus for holding journal |
| CN101221585A (en) * | 2008-02-03 | 2008-07-16 | 华为技术有限公司 | Data storage method and device |
| US20080256309A1 (en) * | 2007-04-11 | 2008-10-16 | Kenneth Wayne Boyd | Maintain owning application information of data for a data storage system |
| CN101534213A (en) * | 2009-04-09 | 2009-09-16 | 成都市华为赛门铁克科技有限公司 | Acquisition method of log and log server |
| CN102164043A (en) * | 2010-10-27 | 2011-08-24 | 青岛海信传媒网络技术有限公司 | Log reporting method and terminal |
| CN102447633A (en) * | 2011-12-29 | 2012-05-09 | 北京亿赞普网络技术有限公司 | Log transmission method and system |
| CN102447742A (en) * | 2011-11-24 | 2012-05-09 | 中兴通讯股份有限公司 | Dynamic data active and standby synchronization method and system as well as metadata server |
-
2012
- 2012-12-26 CN CN201210575837.XA patent/CN103064779B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1932812A (en) * | 2005-09-16 | 2007-03-21 | 腾讯科技（深圳）有限公司 | Method and apparatus for holding journal |
| US20080256309A1 (en) * | 2007-04-11 | 2008-10-16 | Kenneth Wayne Boyd | Maintain owning application information of data for a data storage system |
| CN101221585A (en) * | 2008-02-03 | 2008-07-16 | 华为技术有限公司 | Data storage method and device |
| CN101534213A (en) * | 2009-04-09 | 2009-09-16 | 成都市华为赛门铁克科技有限公司 | Acquisition method of log and log server |
| CN102164043A (en) * | 2010-10-27 | 2011-08-24 | 青岛海信传媒网络技术有限公司 | Log reporting method and terminal |
| CN102447742A (en) * | 2011-11-24 | 2012-05-09 | 中兴通讯股份有限公司 | Dynamic data active and standby synchronization method and system as well as metadata server |
| CN102447633A (en) * | 2011-12-29 | 2012-05-09 | 北京亿赞普网络技术有限公司 | Log transmission method and system |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793517A (en) * | 2014-02-12 | 2014-05-14 | 浪潮电子信息产业股份有限公司 | File system log dump dynamic capacity-increase method based on monitoring mechanism |
| CN105760283A (en) * | 2014-12-18 | 2016-07-13 | 阿里巴巴集团控股有限公司 | Log output method and device |
| CN105760283B (en) * | 2014-12-18 | 2018-11-09 | 阿里巴巴集团控股有限公司 | A kind of log-output method and device |
| CN106951488A (en) * | 2017-03-14 | 2017-07-14 | 海尔优家智能科技（北京）有限公司 | A kind of log recording method and device |
| CN106951488B (en) * | 2017-03-14 | 2021-03-12 | 海尔优家智能科技（北京）有限公司 | Log recording method and device |
| CN108804295A (en) * | 2017-04-28 | 2018-11-13 | 北京京东尚科信息技术有限公司 | log information recording method and device |
| CN111723130A (en) * | 2020-06-28 | 2020-09-29 | 杭州安恒信息技术股份有限公司 | Historical data query method, device and equipment |
| CN112527612A (en) * | 2020-11-06 | 2021-03-19 | 北京罗克维尔斯科技有限公司 | System log preservation method and device, vehicle-mounted system and vehicle |
| CN112527612B (en) * | 2020-11-06 | 2024-01-02 | 北京罗克维尔斯科技有限公司 | System log security method and device, vehicle-mounted system and vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103064779B (en) | 2016-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103064779A (en) | Method for processing log information rapidly | |
| CN104317800A (en) | Hybrid storage system and method for mass intelligent power utilization data | |
| CN103020255B (en) | Classification storage means and device | |
| CN111427844B (en) | Data migration system and method for file hierarchical storage | |
| CN102467412B (en) | Method, device and business system for processing operation request | |
| CN102158349A (en) | Log management device and method thereof | |
| CN105701190A (en) | Data synchronizing method and device | |
| CN101493838B (en) | Event record processing device and system | |
| CN103678603A (en) | Multi-source heterogeneous data efficient converging and storing frame system | |
| CN103077241A (en) | Method for loading data in parallel after splitting files | |
| CN103916459A (en) | Big data filing and storing system | |
| CN103647850A (en) | Data processing method, device and system of distributed version control system | |
| CN106648442A (en) | Metadata node internal memory mirroring method and device | |
| CN106330963A (en) | Cross-network multi-node log collecting method | |
| WO2023185111A1 (en) | Quick access method and device for data file | |
| CN101827120A (en) | Cluster storage method and system | |
| CN105353987A (en) | File processing method and device | |
| CN103294413A (en) | Mass data acquisition terminal supported distributed-memory real-time storage device and storage method | |
| CN102523410B (en) | Method for writing video data and video data storage equipment | |
| CN103778120A (en) | Global file identification generation method, generation device and corresponding distributed file system | |
| CN106161637A (en) | A kind of cloud backup method and device | |
| CN101770504B (en) | Data storage method, data reading method, and data reading equipment | |
| CN102184141A (en) | Method and device for storing check point data | |
| KR101666440B1 (en) | Data processing method in In-memory Database System based on Circle-Queue | |
| CN104077368A (en) | History data two-level caching multi-stage submitting method for dispatching monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |