CN108959497A - distributed file system log processing method, device, equipment and storage medium - Google Patents
distributed file system log processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN108959497A CN108959497A CN201810668179.6A CN201810668179A CN108959497A CN 108959497 A CN108959497 A CN 108959497A CN 201810668179 A CN201810668179 A CN 201810668179A CN 108959497 A CN108959497 A CN 108959497A
- Authority
- CN
- China
- Prior art keywords
- log
- module
- file system
- distributed file
- action type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a kind of distributed file system log processing methods, method includes the following steps: determining the first action type of business operation when monitoring that client has business operation；Determine that corresponding first log module of the first action type, the corresponding log module of different operation type are different；By the operation log recording of pre-generated business operation into the first log module.Using technical solution provided by the embodiment of the present invention, when needing to carry out the lookup of operation log, it can be searched in corresponding log module according to action type, avoid searching from huge journal file, can reduce lookup difficulty, improve search efficiency.The invention also discloses a kind of distributed file system log processing device, equipment and storage mediums, have relevant art effect.
Description
Technical field
The present invention relates to computer application technologies, more particularly to a kind of distributed file system log processing side
Method, device, equipment and storage medium.
Background technique
With the fast development of computer technology, distributed file system using more and more extensive.Client is accessing
When distributed file system, various businesses operation can be carried out to the mass file in system, distributed file system will needle
Various businesses are operated and generate operation log, with for subsequent carry out system maintenance, or carry out problem investigation positioning offer foundation.
Currently, distributed file system is all that operation log is carried out to unified storage.With making for distributed file system
With when there is the lookup demand to operation log, needing from huge operation log wherein the operation log stored is more and more
Middle relevant search information, the lookup time needed is longer, and lookup difficulty is larger, and search efficiency is not high.
Summary of the invention
The object of the present invention is to provide a kind of distributed file system log processing method, device, equipment and storage medium,
To avoid the search operation log from huge journal file, lookup difficulty can reduce, improve search efficiency.
In order to solve the above technical problems, the invention provides the following technical scheme:
A kind of distributed file system log processing method, comprising:
When monitoring that client has business operation, the first action type of the business operation is determined；
Determine corresponding first log module of first action type, the corresponding log module of different operation type is not
Together；
By the operation log recording of the pre-generated business operation into first log module.
In a kind of specific embodiment of the invention, further includes:
When checking aiming at the problem that having the second action type, the corresponding second log mould of second action type is determined
Block；
The search operation log in second log module, it is fixed to carry out problem investigation based on the operation log found
Position.
In a kind of specific embodiment of the invention, first log module is read through model, writing module, removing module
Or modified module.
In a kind of specific embodiment of the invention, first log module includes multiple corresponding with information category
Log submodule, the operation log recording by the pre-generated business operation is into first log module, packet
It includes:
Determine the information category and the corresponding log submodule of every kind of information category that the operation log of the business operation includes
Block；
The information category that operation log based on the business operation includes, by the behaviour of the pre-generated business operation
Make log to be recorded respectively in the corresponding log submodule of first log module.
In a kind of specific embodiment of the invention, the log submodule is error message submodule, warning message
Submodule, prompt information submodule or basic information submodule.
A kind of distributed file system log processing device, comprising:
Action type determining module, for when monitoring that client has business operation, determining the of the business operation
One action type；
Log module determining module, for determining corresponding first log module of first action type, different operation
The corresponding log module of type is different；
Operation log recording module, for by the operation log recording of the business operation pre-generated to described first
In log module.
In a kind of specific embodiment of the invention, further includes operation log searching module, is used for:
When checking aiming at the problem that having the second action type, the corresponding second log mould of second action type is determined
Block；
The search operation log in second log module, it is fixed to carry out problem investigation based on the operation log found
Position.
In a kind of specific embodiment of the invention, first log module includes multiple corresponding with information category
Log submodule, the operation log recording module, is specifically used for:
Determine the information category and the corresponding log submodule of every kind of information category that the operation log of the business operation includes
Block；
The information category that operation log based on the business operation includes, by the behaviour of the pre-generated business operation
Make log to be recorded respectively in the corresponding log submodule of first log module.
A kind of distributed file system log processing equipment, comprising:
Memory, for storing computer program；
Processor is realized at distributed file system log described in any of the above-described when for executing the computer program
The step of reason method.
A kind of computer readable storage medium is stored with computer program on the computer readable storage medium, described
The step of distributed file system log processing method described in any of the above-described is realized when computer program is executed by processor.
Using technical solution provided by the embodiment of the present invention, when monitoring that client has business operation, can determine
First action type of business operation, and determine corresponding first log module of the first action type, by pre-generated business
The operation log recording of operation is into the first log module, business operation corresponding to the operation log of every kind of log module record
Action type it is consistent, can be according to action type in corresponding log mould in this way, when needing to carry out the lookup of operation log
It is searched in block, avoids searching from huge journal file, can reduce lookup difficulty, improve search efficiency.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of implementation flow chart of distributed file system log processing method in the embodiment of the present invention；
Fig. 2 is a kind of structural schematic diagram of distributed file system log processing device in the embodiment of the present invention；
Fig. 3 is a kind of structural schematic diagram of distributed file system log processing equipment in the embodiment of the present invention.
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, with reference to the accompanying drawings and detailed description
The present invention is described in further detail.Obviously, described embodiments are only a part of the embodiments of the present invention, rather than
Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise
Under every other embodiment obtained, shall fall within the protection scope of the present invention.
It is shown in Figure 1, it is a kind of reality of distributed file system log processing method provided by the embodiment of the present invention
Flow chart is applied, this method may comprise steps of:
S110: when monitoring that client has business operation, the first action type of business operation is determined.
In practical applications, distributed file system can be to distributed document with carry multiple client, client
Mass file in system reads and writes equal business operations.Distributed file system the business to each client can be grasped in real time
It is monitored.
Each business operation has corresponding operation type.The action type of business operation can divide in advance, such as operate
Type can be the types such as reading and writing, deletion, modification, be further divided into the types such as setting, inquiry, acquisition.Monitoring client
When having business operation, the first action type of business operation can be determined.Such as, the first action type of business operation is determined to read
Type.Here client is any one client of carry on a distributed.
S120: corresponding first log module of the first action type is determined.
Wherein, the corresponding log module of different operation type is different.
In embodiments of the present invention, there are many log modules for distributed file system setting, such as read through model, writing module, delete
Except module, modified module etc., different log modules are distinguished using not isolabeling.The corresponding log module of different operation type
Difference, a kind of action type correspond to a kind of log module.Specifically, reading type corresponds to read through model, writes type and correspond to writing module, delete
Except type corresponds to removing module, modification type corresponds to modified module.
After determining the first action type of business operation, it may further determine that the first action type is first day corresponding
Will module.First log module can be read through model, writing module, removing module or modified module.
S130: by the operation log recording of pre-generated business operation into the first log module.
It in practical applications, can be for client when distributed file system monitors that client has business operation
Each business operation generates corresponding operation log.It can recorde operating time, operation object, operating result etc. in operation log
Information.
It should be noted that the step of operation log of the embodiment of the present invention for generating business operation and the business of determination behaviour
The execution sequence of the step of the first action type made is with no restrictions.
Determining corresponding first log module of the first action type, it, can will after the operation log for generating business operation
The operation log recording of business operation is into the first log module, so that record has the first action type in the first log module
Business operation operation log, i.e. every kind of log module record operation log corresponding to business operation action type all
It is consistent.To the operation log differentiation processing in different log modules, convenient for the tracking and when something goes wrong of business I O process
Investigation positioning.
In a kind of specific embodiment of the invention, the first log module may include multiple corresponding with information category
Log submodule, step S130 may comprise steps of:
First step: determine that information category that the operation log of business operation includes and every kind of information category are corresponding
Log submodule；
Second step: the information category that the operation log based on business operation includes, by pre-generated business operation
Operation log be recorded in the corresponding log submodule of the first log module respectively.
It is illustrated for ease of description, above-mentioned two step is combined.
It is understood that there are many information of the operation log recording of a business operation is possible, such as error message, police
Accuse information, prompt information, basic information etc..Each log module may include multiple log submodules corresponding with information category
Block.
After the operation log for generating business operation, the information category that the operation log of business operation includes can be determined,
Such as wrong class, warning class, further determine the corresponding log submodule of every kind of information category.Log submodule is mistake letter
Cease submodule, warning message submodule, prompt information submodule or basic information submodule.That is the corresponding log submodule of mistake class
Block is error message submodule, and the corresponding log submodule of warning class is warning message submodule, prompts the corresponding day measure of class
Module is prompt information submodule, and the corresponding log submodule of foundation class is basic message sub-module.
After determining corresponding first log module of the first action type, information that the operation log based on business operation includes
The corresponding log submodule of the first log module can be recorded in the operation log of pre-generated business operation by classification respectively
It, can be based on the information category of particular problem, in a corresponding day measure in this way when needing to carry out the lookup of operation log in block
It is searched in module, further increases the search efficiency of operation log.
Using method provided by the embodiment of the present invention, when monitoring that client has business operation, business can be determined
First action type of operation, and determine corresponding first log module of the first action type, by pre-generated business operation
Operation log recording into the first log module, every kind of log module record operation log corresponding to business operation behaviour
Make that type is consistent, in this way, when needing to carry out the lookup of operation log, it can be according to action type in corresponding log module
It searches, avoids searching from huge journal file, can reduce lookup difficulty, improve search efficiency.
In one embodiment of the invention, this method may also comprise the following steps::
Step 1: when checking aiming at the problem that having the second action type, the corresponding second day of the second action type is determined
Will module；
Step 2: the search operation log in the second log module, to carry out problem row based on the operation log found
Vouch position.
It is illustrated for ease of description, above-mentioned two step is combined.
In embodiments of the present invention, the action type of business operation corresponding to the operation log of every kind of log module record
Unanimously.When checking aiming at the problem that having the second action type, the corresponding second log mould of the second action type can be first determined
Block.Record has the operation log of the business operation with the second action type in second log module.Determine the second log module
Afterwards, the second log module can be opened, corresponding operation log is searched in the second log module, it in this way can be based on lookup
Operation log carries out problem investigation positioning, does not need the relevant search information from huge journal file, can reduce problem row
Look into difficulty, quick positioning question.
The embodiment of the present invention carries out the record of operation log by a variety of log modules, can quick analyzing and positioning problem,
The stability for improving distributed file system improves the high availability of storage, reduces system maintenance cost.
Corresponding to above method embodiment, the embodiment of the invention also provides a kind of distributed file system log processings
Device, a kind of distributed file system log processing device described below and a kind of above-described distributed file system day
Will processing method can correspond to each other reference.
Shown in Figure 2, which comprises the following modules:
Action type determining module 210, for when monitoring that client has business operation, determining the first of business operation
Action type；
Log module determining module 220, for determining corresponding first log module of the first action type, different operation class
The corresponding log module of type is different；
Operation log recording module 230, for by the operation log recording of the business operation pre-generated to the first log
In module.
Using device provided by the embodiment of the present invention, when monitoring that client has business operation, business can be determined
First action type of operation, and determine corresponding first log module of the first action type, by pre-generated business operation
Operation log recording into the first log module, every kind of log module record operation log corresponding to business operation behaviour
Make that type is consistent, in this way, when needing to carry out the lookup of operation log, it can be according to action type in corresponding log module
It searches, avoids searching from huge journal file, can reduce lookup difficulty, improve search efficiency.
In a kind of specific embodiment of the invention, further includes operation log searching module, is used for:
When checking aiming at the problem that having the second action type, corresponding second log module of the second action type is determined；
The search operation log in the second log module, to carry out problem investigation positioning based on the operation log found.
In a kind of specific embodiment of the invention, the first log module is read through model, writing module, removing module or repairs
Change module.
In a kind of specific embodiment of the invention, the first log module includes multiple logs corresponding with information category
Submodule, operation log recording module 230, is specifically used for:
Determine the information category and the corresponding log submodule of every kind of information category that the operation log of business operation includes；
The information category that operation log based on business operation includes, by the operation log point of pre-generated business operation
It is not recorded in the corresponding log submodule of the first log module.
In a kind of specific embodiment of the invention, log submodule is error message submodule, warning message submodule
Block, prompt information submodule or basic information submodule.
Corresponding to above method embodiment, the embodiment of the invention also provides a kind of distributed file system log processings
Equipment, it is shown in Figure 3, comprising:
Memory 310, for storing computer program；
Processor 320 realizes the step of above-mentioned distributed file system log processing method when for executing computer program
Suddenly.
Corresponding to above method embodiment, the embodiment of the invention also provides a kind of computer readable storage medium, meters
It is stored with computer program on calculation machine readable storage medium storing program for executing, above-mentioned distributed document is realized when computer program is executed by processor
The step of system log processing method.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with it is other
The difference of embodiment, same or similar part may refer to each other between each embodiment.For being filled disclosed in embodiment
For setting, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method part
Explanation.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Used herein a specific example illustrates the principle and implementation of the invention, and above embodiments are said
It is bright to be merely used to help understand technical solution of the present invention and its core concept.It should be pointed out that for the common of the art
, without departing from the principle of the present invention, can be with several improvements and modifications are made to the present invention for technical staff, these
Improvement and modification are also fallen within the protection scope of the claims of the present invention.
Claims (10)
1. a kind of distributed file system log processing method characterized by comprising
When monitoring that client has business operation, the first action type of the business operation is determined；
Determine that corresponding first log module of first action type, the corresponding log module of different operation type are different；
By the operation log recording of the pre-generated business operation into first log module.
2. distributed file system log processing method according to claim 1, which is characterized in that further include:
When checking aiming at the problem that having the second action type, corresponding second log module of second action type is determined；
The search operation log in second log module, to carry out problem investigation positioning based on the operation log found.
3. distributed file system log processing method according to claim 1, which is characterized in that the first log mould
Block is read through model, writing module, removing module or modified module.
4. according to claim 1 to described in any item distributed file system log processing methods among 3, which is characterized in that
First log module includes multiple log submodules corresponding with information category, described to grasp the pre-generated business
The operation log recording of work is into first log module, comprising:
Determine the information category and the corresponding log submodule of every kind of information category that the operation log of the business operation includes；
The information category that operation log based on the business operation includes, by the operation day of the pre-generated business operation
Will is recorded respectively in the corresponding log submodule of first log module.
5. distributed file system log processing method according to claim 4, which is characterized in that the log submodule
For error message submodule, warning message submodule, prompt information submodule or basic information submodule.
6. a kind of distributed file system log processing device characterized by comprising
Action type determining module, for when monitoring that client has business operation, determining the first behaviour of the business operation
Make type；
Log module determining module, for determining corresponding first log module of first action type, different operation type
Corresponding log module is different；
Operation log recording module, for by the operation log recording of the business operation pre-generated to first log
In module.
7. distributed file system log processing device according to claim 6, which is characterized in that further include operation log
Searching module is used for:
When checking aiming at the problem that having the second action type, corresponding second log module of second action type is determined；
The search operation log in second log module, to carry out problem investigation positioning based on the operation log found.
8. distributed file system log processing device according to claim 6 or 7, which is characterized in that described first day
Will module includes multiple log submodules corresponding with information category, and the operation log recording module is specifically used for:
Determine the information category and the corresponding log submodule of every kind of information category that the operation log of the business operation includes；
The information category that operation log based on the business operation includes, by the operation day of the pre-generated business operation
Will is recorded respectively in the corresponding log submodule of first log module.
9. a kind of distributed file system log processing equipment characterized by comprising
Memory, for storing computer program；
Processor realizes the distributed file system as described in any one of claim 1 to 5 when for executing the computer program
The step of log processing method.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes the distributed file system day as described in any one of claim 1 to 5 when the computer program is executed by processor
The step of will processing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810668179.6A CN108959497A (en) | 2018-06-26 | 2018-06-26 | distributed file system log processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810668179.6A CN108959497A (en) | 2018-06-26 | 2018-06-26 | distributed file system log processing method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108959497A true CN108959497A (en) | 2018-12-07 |
Family
ID=64486639
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810668179.6A Pending CN108959497A (en) | 2018-06-26 | 2018-06-26 | distributed file system log processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108959497A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851340A (en) * | 2019-11-06 | 2020-02-28 | 香港乐蜜有限公司 | Operation log collection method and device and server |
| CN111062593A (en) * | 2019-12-06 | 2020-04-24 | 深圳供电局有限公司 | Distribution network operation management method and system, computer equipment and storage medium |
| CN113885809A (en) * | 2021-12-07 | 2022-01-04 | 云和恩墨(北京)信息技术有限公司 | Data management system and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103631699A (en) * | 2012-08-28 | 2014-03-12 | 纽海信息技术（上海）有限公司 | Log management system and method for log monitoring, acquiring and querying |
| CN103793479A (en) * | 2014-01-14 | 2014-05-14 | 上海上讯信息技术股份有限公司 | Log management method and log management system |
| CN104935469A (en) * | 2015-06-30 | 2015-09-23 | 浪潮(北京)电子信息产业有限公司 | Distributive storage method and system for log information |
| CN108052675A (en) * | 2017-12-28 | 2018-05-18 | 惠州Tcl家电集团有限公司 | Blog management method, system and computer readable storage medium |
-
2018
- 2018-06-26 CN CN201810668179.6A patent/CN108959497A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103631699A (en) * | 2012-08-28 | 2014-03-12 | 纽海信息技术（上海）有限公司 | Log management system and method for log monitoring, acquiring and querying |
| CN103793479A (en) * | 2014-01-14 | 2014-05-14 | 上海上讯信息技术股份有限公司 | Log management method and log management system |
| CN104935469A (en) * | 2015-06-30 | 2015-09-23 | 浪潮(北京)电子信息产业有限公司 | Distributive storage method and system for log information |
| CN108052675A (en) * | 2017-12-28 | 2018-05-18 | 惠州Tcl家电集团有限公司 | Blog management method, system and computer readable storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851340A (en) * | 2019-11-06 | 2020-02-28 | 香港乐蜜有限公司 | Operation log collection method and device and server |
| CN111062593A (en) * | 2019-12-06 | 2020-04-24 | 深圳供电局有限公司 | Distribution network operation management method and system, computer equipment and storage medium |
| CN113885809A (en) * | 2021-12-07 | 2022-01-04 | 云和恩墨(北京)信息技术有限公司 | Data management system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103136243B (en) | File system duplicate removal method based on cloud storage and device | |
| CN103019890B (en) | Block-level disk data protection system and method thereof | |
| CN109542342B (en) | Metadata management and data reconstruction method, equipment and storage medium | |
| US9996557B2 (en) | Database storage system based on optical disk and method using the system | |
| US20160055189A1 (en) | Database Storage System based on Optical Disk Library, and Method Using Same | |
| CN110046168A (en) | A kind of incremental data consistency implementation method and device | |
| CN108959497A (en) | distributed file system log processing method, device, equipment and storage medium | |
| Levin et al. | AIOps for a cloud object storage service | |
| CN102779138B (en) | The hard disk access method of real time data | |
| CN109491589A (en) | A kind of delamination process and device based on file content | |
| CN107665219A (en) | A kind of blog management method and device | |
| CN109376046A (en) | A kind of method for testing hard disk, device, equipment and computer readable storage medium | |
| CN104461384B (en) | A kind of method for writing data and storage device | |
| CN106021052A (en) | GJB289A bus testing system and method | |
| CN106354587A (en) | Mirror image server and method for exporting mirror image files of virtual machine | |
| CN112463880B (en) | Block chain data storage method and related device | |
| CN107515807A (en) | A kind of method and device for storing monitoring data | |
| CN109446167A (en) | A kind of storage of daily record data, extracting method and device | |
| CN111435327B (en) | Log record processing method, device and system | |
| US7028041B2 (en) | Methods and apparatus for database space calculation and error detection | |
| CN105242985B (en) | Data recovery method and device | |
| CN115640158A (en) | Detection analysis method and device based on database | |
| CN112799951B (en) | Method and equipment for generating automatic test case for charging system | |
| CN107330002A (en) | Distributed data base implementation method and device | |
| CN105279103A (en) | Data management method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181207 |