CN108170578A - log collecting method and device - Google Patents
log collecting method and device Download PDFInfo
- Publication number
- CN108170578A CN108170578A CN201810015655.4A CN201810015655A CN108170578A CN 108170578 A CN108170578 A CN 108170578A CN 201810015655 A CN201810015655 A CN 201810015655A CN 108170578 A CN108170578 A CN 108170578A
- Authority
- CN
- China
- Prior art keywords
- log
- module
- path
- message queue
- output module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 85
- 238000012545 processing Methods 0.000 claims description 37
- 230000015654 memory Effects 0.000 description 18
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 4
- 238000004321 preservation Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Debugging And Monitoring (AREA)
Abstract
Log collecting method and device provided by the invention, are related to journaling technique field.Wherein, log collecting method includes：Obtain the log information that journal file generates, log information is packaged to obtain log event, and log event is sent to corresponding output module, wherein, log event includes routing information, when connecting in container in the presence of log path matched with the routing information, the log event in the output module is sent to by the message queue Categories Address by the connection between the log path and message queue Categories Address.It is realized by the above method and exports the log information that different journal files generates to different message queue Categories Addresses, in order to which the log information generated to different journal files is managed and uses.
Description
Technical field
The present invention relates to journaling technique field, in particular to a kind of log collecting method and device.
Background technology
In existing production environment, when being collected to the log information that journal file generates, log information can be pressed
According to being saved in certain format print to file, when needing these daily records carrying out subsequent processing, typically
It, can be from the message queue in subsequent processing procedure by being output in a message queue for these daily record a line a line
Middle acquisition daily record, and be managed and use.Through inventor the study found that there is management using above-mentioned processing method and use
The problem of inconvenient.
Invention content
In view of this, the purpose of the present invention is to provide a kind of log collecting method and devices, realize different daily records
The log information that file generates is exported to different message queue Categories Addresses, in order to the daily record generated to different journal files
Information is managed and uses.
To achieve the above object, the embodiment of the present invention adopts the following technical scheme that：
A kind of log collecting method, including：
The log information that journal file generates is obtained, the log information is packaged to obtain log event, and will
The log event is sent to corresponding output module, wherein, different journal files is corresponding with different output modules, and institute
It states log event and includes routing information；
Judge connection container in whether there is with the matched log path of the routing information, wherein, the connection container
Include the connection between multiple log paths and the multiple log path and corresponding message queue Categories Address；
When connecting in container in the presence of log path matched with the routing information, pass through the log path and message team
Log event in the output module is sent to the message queue Categories Address by the connection between row Categories Address.
Optionally, it in above-mentioned log collecting method, is not present and the routing information matched day when connecting in container
During will path, the method further includes：
Corresponding with routing information log path is searched from preset database and the log path is corresponding disappears
Queue Categories Address is ceased, create the connection between the log path and the corresponding message queue Categories Address and is preserved to described
It connects in container, wherein, it is stored in the database corresponding between different log paths and message queue Categories Address
Relationship；
By the connection between the log path of establishment and corresponding message queue Categories Address by the output mould
Log event in the block is sent to the message queue Categories Address.
Optionally, it in above-mentioned log collecting method, is performing the log information for obtaining journal file generation and is being sealed
Before the step of being filled with to obtain log event, the method further includes：
The output module and corresponding instantiation procedure are created in log collection tool and is preserved into data dictionary,
Wherein, the corresponding instantiation procedure of different output modules is different；
The title of corresponding output module is searched from the data dictionary according to the corresponding module title of journal file, and
Connection container is created according to the corresponding instantiation procedure of the title of output module and journal file.
Optionally, in above-mentioned log collecting method, the step of log event is sent to corresponding output module
Including：
Call the corresponding output module of the log event interface method the log event is sent to it is corresponding defeated
Go out module.
Optionally, in above-mentioned log collecting method, the output module is defeated for file output processing module or database
Go out processing module.
The present invention also provides a kind of log collection device, described device includes：
Log processing module, for obtain journal file generation log information, by the log information be packaged with
Log event is obtained, and the log event is sent to corresponding output module, wherein, different journal files is corresponding with not
Same output module, and the log event includes routing information；
Judgment module, for judge connect container in whether there is with the matched log path of the routing information, wherein,
The connection container include multiple log paths and the multiple log path and corresponding message queue Categories Address it
Between connection；
Sending module, for when there is log path matched with the routing information in connecting container, passing through the day
Log event in the output module is sent to the message queue by the connection between will path and message queue Categories Address
Categories Address.
Optionally, it in above-mentioned log collection device, is not present and the routing information matched day when connecting in container
During will path, described device further includes：
Connection establishment module, for searching log path corresponding with the routing information from preset database and being somebody's turn to do
The corresponding message queue Categories Address of log path is created between the log path and the corresponding message queue Categories Address
It connects and preserves into the connection container, wherein, different log paths and message queue point are stored in the database
Correspondence between class address；
The sending module is additionally operable to by between the log path of establishment and corresponding message queue Categories Address
Connection the log event in the output module is sent to the message queue Categories Address.
Optionally, in above-mentioned log collection device, described device further includes：
First creation module, for creating the output module and corresponding instantiation procedure in log collection tool simultaneously
It preserves into data dictionary, wherein, the corresponding instantiation procedure of different output modules is different；
Second creation module, it is corresponding for being searched from the data dictionary according to the corresponding module title of journal file
The title of output module, and connection container is created according to the corresponding instantiation procedure of title and journal file of output module.
Optionally, in above-mentioned log collection device, the log processing module is additionally operable to call the log event pair
The log event is sent to corresponding output module by the interface method for the output module answered.
Optionally, in above-mentioned log collection device, the output module is defeated for file output processing module or database
Go out processing module.
A kind of log collecting method and device provided by the invention, the log information generated by obtaining journal file will
Log information is packaged to obtain log event, and log event is sent to corresponding output module, wherein, log event
Include routing information, when connecting in container in the presence of log path matched with the routing information, pass through the log path
Log event in the output module is sent to the message queue taxonomically by the connection between message queue Categories Address
Location.It is realized by the above method and exports the log information that different journal files generates to different message queues taxonomically
Location, in order to which the log information generated to different journal files is managed and uses.
For the above objects, features and advantages of the present invention is enable to be clearer and more comprehensible, preferred embodiment cited below particularly, and coordinate
Appended attached drawing, is described in detail below.
Description of the drawings
Fig. 1 is the structure diagram of terminal device provided in an embodiment of the present invention.
Fig. 2 is the flow diagram of log collecting method provided in an embodiment of the present invention.
Fig. 3 is another flow diagram of log collecting method provided in an embodiment of the present invention.
Fig. 4 is another flow diagram of log collecting method provided in an embodiment of the present invention.
Fig. 5 is the connection block diagram of log collection device provided in an embodiment of the present invention.
Fig. 6 is another connection block diagram of log collection device provided in an embodiment of the present invention.
Icon：10- terminal devices；12- memories；14- processors；100- log collection devices；110- log processing moulds
Block；120- judgment modules；130- sending modules；140- connection establishment modules；The first creation modules of 160-；170- second creates mould
Block.
Specific embodiment
Purpose, technical scheme and advantage to make the embodiment of the present invention are clearer, below in conjunction with the embodiment of the present invention
In attached drawing, the technical solution in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment only
It is the part of the embodiment of the present invention, instead of all the embodiments.The present invention being usually described and illustrated herein in the accompanying drawings
The component of embodiment can be configured to arrange and design with a variety of different.
Therefore, below the detailed description of the embodiment of the present invention to providing in the accompanying drawings be not intended to limit it is claimed
The scope of the present invention, but be merely representative of the present invention selected embodiment.Based on the embodiments of the present invention, this field is common
Technical staff's all other embodiments obtained without creative efforts belong to the model that the present invention protects
It encloses.
It should be noted that：Similar label and letter represents similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need to that it is further defined and explained in subsequent attached drawing.
As shown in Figure 1, an embodiment of the present invention provides a kind of terminal device 10, including memory 12, processor 14 and day
Will collection device 100.Wherein, the terminal device 10 can include, but are not limited to be server, smart mobile phone, PC
(personal computer, PC), tablet computer, personal digital assistant (personal digital assistant, PDA),
Mobile internet surfing equipment (mobile Internet device, MID) etc..
In this embodiment, it is directly or indirectly electrically connected between the memory 12 and processor 14, to realize
The transmission or interaction of data.For example, these elements can realize electricity by one or more communication bus or signal wire between each other
Property connection.The log collection device 100 can be stored in institute including at least one in the form of software or firmware (firmware)
State the software function module in memory 12.The processor 14 is used to perform the executable mould stored in the memory 12
Block, such as software function module included by the log collection device 100 and computer program etc., to realize log collection side
Method.
Wherein, the memory 12 may be, but not limited to, random access memory (Random Access Memory,
RAM), read-only memory (Read Only Memory, ROM), programmable read only memory (Programmable Read-Only
Memory, PROM), erasable read-only memory (Erasable Programmable Read-Only Memory, EPROM),
Electricallyerasable ROM (EEROM) (Electric Erasable Programmable Read-Only Memory, EEPROM) etc..
Wherein, for memory 12 for storing program, the processor 14 performs described program after execute instruction is received.
The processor 14 may be a kind of IC chip, have the processing capacity of signal.Above-mentioned processor 14
Can be general processor, including central processing unit (Central Processing Unit, CPU), network processing unit
(Network Processor, NP) etc.；It can also be digital signal processor (DSP), application-specific integrated circuit (ASIC), scene
Programmable gate array (FPGA) either other programmable logic device, discrete gate or transistor logic, discrete hardware group
Part.It can realize or perform disclosed each method, step and the logic diagram in the embodiment of the present invention.General processor can be with
It is microprocessor or the processor can also be any conventional processor etc..
It is appreciated that structure shown in FIG. 1 is only to illustrate, the terminal device 10 may also include more than shown in Fig. 1
Either less component or with the configuration different from shown in Fig. 1.Hardware, software may be used in each component shown in Fig. 1
Or combination is realized.
With reference to Fig. 2, the embodiment of the present invention also provides a kind of log collecting method that can be applied to above-mentioned terminal device 10,
It the described method comprises the following steps tri- steps of S110- steps S130：
Step S110：The log information that journal file generates is obtained, the log information is packaged to obtain daily record
Event, and the log event is sent to corresponding output module.
Wherein, different journal files is corresponding with different output modules, and the log event includes routing information.
The mode for obtaining the log information that the journal file generates is specially：Client is installed on the terminal device 10
Afterwards, each journal file in journal file collection kit (filebeat) the monitor terminal equipment 10 in terminal device 10, tracking
These journal files are read to obtain the log information of journal file generation.Different journal files is corresponding with different path letters
Breath, when obtaining the log information, is sent to after the corresponding routing information of the journal file and log information are packaged
Corresponding output module.
In the present embodiment, the mode for the log event being sent to corresponding output module can be：Described in calling
The log event is sent to corresponding output module by the interface method of the corresponding output module of log event.
Specifically, the output module can realize the effect of Client interfaces, and the characteristic for having object-oriented polymorphic,
Above-mentioned interface method is to realize the above-mentioned effect with Client interfaces, and the method with the polymorphic characteristic of object-oriented is with general
The log event is sent to corresponding output module.
Step S120：Judge to whether there is and the matched log path of the routing information in connection container.
Wherein, the connection container includes multiple log paths and the multiple log path and corresponding message
Connection between queue Categories Address.
The log path is the store path of log information, and the message queue Categories Address is one in message system
The corresponding address of a specification area, the connection container are pre-stored, and in the connection container multiple log paths with
Connection between corresponding message queue Categories Address can pre-establish.
Step S130：When connecting in container in the presence of log path matched with the routing information, pass through the daily record road
Log event in the output module is sent to message queue classification by the connection between diameter and message queue Categories Address
Address.
By above-mentioned setting, with when the log information to different journal file generations stores, realizing will be different
The log information that generates of journal file be sent at different message queue Categories Addresses, and then realize different daily record texts
The log information correspondence that part generates stores the different classifications region into message system, in order to what is generated to different journal files
Log information is managed and uses.
In addition, effectively avoided using existing log collecting method by using above method step, by daily record a line
After being output in a message queue of a line, if you need to which the log content in message queue is output to some classification specified
The inside needs to specify a classification for the problem that each journal file path, caused by cost it is excessively high and development efficiency is low.
Incorporated by reference to Fig. 3, optionally, in the present embodiment, when connecting in container, there is no matched with the routing information
During log path, the method further includes：
Step S140：Log path corresponding with the routing information and the log path are searched from preset database
Corresponding message queue Categories Address creates the connection between the log path and corresponding message queue Categories Address and preservation
Into the connection container.
Wherein, the corresponding pass between different log paths and message queue Categories Address is stored in the database
System.
Step S150：By the connection between the log path of establishment and corresponding message queue Categories Address by institute
It states the log event in output module and is sent to message queue Categories Address.
It is arranged on by above-mentioned there is no during log path matched with the routing information, creates and preserve the daily record road
Connection between diameter and the message queue Categories Address, with subsequently carry out log information collection when, without creating the day again
Connection between will path and the message queue Categories Address, improves the efficiency of log collection, and further realize will be different
The log information that generates of journal file be sent at different message queue Categories Addresses, realize and produce different journal file
Raw log information correspondence stores the different classifications region into message system, in order to the daily record generated to different journal files
Information is managed and uses.
It is appreciated that the output module is the data processor in journal file collection kit.For to receiving
Log event handled and forwarded.The output module can be but not limited to file output processing module or database is defeated
Go out processing module.
Optionally, in the present embodiment, the output module is file output processing module or database output processing mould
Block.It is appreciated that when the output module is multiple, file data processing module can be included in multiple output modules
Quantity and the quantity of data output processing module be not specifically limited herein.
It should be noted that when the output module is file output processing module, pass through the daily record road of establishment
Log event in the output module is sent to message queue by the connection between diameter and corresponding message queue Categories Address
The mode of Categories Address is specifically, pass through the connection between the log path of establishment and corresponding message queue Categories Address
Log event in the file output processing module is sent in the corresponding file of message queue Categories Address.When described defeated
When going out module for database output processing module, by the log path and corresponding message queue Categories Address of establishment it
Between connection the log event in the output module is sent to the mode of message queue Categories Address specifically, passing through establishment
The log path and corresponding message queue Categories Address between connection will be in the database output processing module
Log event is sent in the corresponding database of message queue Categories Address.
Incorporated by reference to Fig. 4, optionally, in the present embodiment, the step of obtaining log information that journal file generates is being performed
Before, the method further includes：
Step S160：The output module and corresponding instantiation procedure are created in log collection tool and is preserved to number
According in dictionary.
Wherein, the corresponding instantiation procedure of different output modules is different.
The mode of the output module and corresponding instantiation procedure is created in the log collection tool to be, to
Configuration information is inputted in the log collection tool, the configuration information includes multiple instantiation procedures and corresponding multiple outputs
The title of module.When starting the log collection tool, the log collection tool is initialized, each instantiation procedure
The title of output module is instantiated to obtain each instantiation procedure and corresponding output module and preserve.Pass through
Above-mentioned setting is to realize the self-defined output module, so that when carrying out log collection, it is independently fixed according to actual demand to realize
Justice, and then realize that the log information generated according to actual demand to different journal files is classified.
Step S170：Corresponding output module is searched from the data dictionary according to the corresponding module title of journal file
Title, and according to the corresponding instantiation procedure of title and journal file of output module create connection container.
It is wrapped specifically, being created in connection container according to the corresponding instantiation procedure of title and journal file of output module
It includes：The log path that the log information generated in the corresponding journal file of each output module obtains after being packaged with it is corresponding
Message queue Categories Address between connection.
In the present embodiment, journal file collection kit (filebeat) is before log collection is carried out, due to
Filebeat is write using go language, can be called based on init () function in go language prior to main () function
Characteristic, therefore the registration logic of processing module is write on inside init () function, complete registering functional.Specifically：It can create
One data word allusion quotation creates a file in this packet and inside writes for storing whole codes of output module
Init () initialization function, to realize to filebeat location registration process module title and instantiation procedure and preserve to data word
In allusion quotation, journal file collection kit is initialized later to obtain processing module and corresponding instantiation procedure, wherein, often
A output module all has the effect for realizing Client interfaces, and with the polymorphic characteristic of object-oriented, for output journal
Event in classifying for the message queue for being output to log event, needs a company corresponded to message queue between classification
It connects, therefore, searches the title of corresponding output module from the data dictionary according to the corresponding module title of journal file, and
Connection container (connectionHolder) is created according to the corresponding instantiation procedure of the title of output module and journal file.With
Connection between multiple log paths and the multiple log path and corresponding message queue Categories Address is preserved to even
It connects in container.In the log information generated in getting journal file, the log information is packaged to obtain daily record
Event is simultaneously sent to corresponding output module, and judges to connect in container and whether there is and the matched daily record road of the routing information
Diameter, when it is present, by the connection between the log path and message queue Categories Address by the daily record in the output module
Event is sent to the message queue Categories Address.When it be not present, it is searched and the routing information pair from preset database
The log path answered and the corresponding message queue Categories Address of the log path, create the log path and the corresponding message team
Connection between row Categories Address is simultaneously preserved into the connection container, with realize by the log path created with it is corresponding
Message queue Categories Address between connection the log event in the output module is sent to the message queue taxonomically
Location, and then complete the collection of log information generated to journal file.
Incorporated by reference to Fig. 5, on the basis of the above, the present invention also provides a kind of log collection device 100, the log collection fills
It puts 100 and includes log processing module 110, judgment module 120 and sending module 130.
The log processing module 110 is used to obtain the log information of journal file generation, and the log information is carried out
The log event is sent to corresponding output module by encapsulation to obtain log event, wherein, different journal files pair
There should be different output modules, and the log event includes routing information.In the present embodiment, the log processing module
110 can be used for performing step S110 shown in Fig. 2, and the specific descriptions about the log processing module 110 are referred to above
Description to step S110.
Optionally, in the present embodiment, the log processing module 110 is additionally operable to call the log event corresponding
The log event is sent to corresponding output module by the interface method of output module.
Optionally, in the present embodiment, the output module is file output processing module or database output processing mould
Block.
The judgment module 120 is used to judge to whether there is in connection container and the matched daily record road of the routing information
Diameter, wherein, the connection container includes multiple log paths and the multiple log path and corresponding message queue point
Connection between class address.In the present embodiment, the judgment module 120 can be used for performing step S120 shown in Fig. 2, about
The specific descriptions of the judgment module 120 are referred to the description to step S120 above.
The sending module 130 is used to, when there is log path matched with the routing information in connecting container, lead to
Log event in the output module is sent to this and disappeared by the connection crossed between the log path and message queue Categories Address
Cease queue Categories Address.In the present embodiment, the sending module 130 can be used for performing step S130 shown in Fig. 2, about institute
The specific descriptions for stating sending module 130 are referred to the description to step S130 above.
Incorporated by reference to Fig. 6, optionally, in the present embodiment, the log collection device 100 further includes connection establishment module
140。
The connection establishment module 140 is used to search daily record road corresponding with the routing information from preset database
Diameter and the corresponding message queue Categories Address of the log path, create the log path and the corresponding message queue Categories Address
Between connection and preserve to it is described connection container in, wherein, different log paths and message are stored in the database
Correspondence between queue Categories Address.In the present embodiment, the connection establishment module 140 can be used for performing shown in Fig. 3
Step S140, the specific descriptions about the connection establishment module 140 are referred to the description to step S140 above.
The sending module 130 be additionally operable to by the log path and corresponding message queue Categories Address of establishment it
Between connection the log event in the output module is sent to the message queue Categories Address.
Optionally, in the present embodiment, the log collection device 100 further includes：First creation module 160 and the second wound
Model block 170.
First creation module 160, for creating the output module and corresponding example in log collection tool
Change program and preserve into data dictionary, wherein, the corresponding instantiation procedure of different output modules is different.In the present embodiment
In, first creation module 160 can be used for performing step S150 shown in Fig. 4, the tool about first creation module 160
Body description is referred to the description to step S150 above.
Second creation module 170, for being looked into from the data dictionary according to the corresponding module title of journal file
The title of corresponding output module is looked for, and is created and connected according to the corresponding instantiation procedure of title and journal file of output module
Container.In the present embodiment, second creation module 170 can be used for performing step S160 shown in Fig. 4, about described second
The specific descriptions of creation module 170 are referred to the description to step S160 above.
To sum up, a kind of log collecting method and device provided by the invention, the log collecting method is by obtaining daily record
The log information that file generates, log information is packaged to obtain log event, and log event is sent to corresponding
Output module, wherein, log event includes routing information, exists and the matched daily record of the routing information when connecting in container
During path, the log event in the output module is sent out by the connection between the log path and message queue Categories Address
It send, to the message queue Categories Address, to realize and exports the log information of different journal file generations to different message teams
Row Categories Address, in order to which the daily record generated to different journal files is managed and uses.Further, in container is connected
There is no during log path matched with the routing information, create and preserve the log path and the message queue Categories Address
Between connection and preservation, with subsequently carry out log information preservation when, without creating the log path and the message team again
Connection between row Categories Address improves the efficiency of log collection, and further realizes different journal file generations
Log information is sent at different message queue Categories Addresses, realizes that the log information by different journal file generations corresponds to
The different classifications region into message system is stored, in order to which the log information generated to different journal files is managed and is made
With.
In the several embodiments provided in the embodiment of the present invention, it should be understood that disclosed device and method also may be used
To realize by another way.Device and method embodiment described above is only schematical, for example, in attached drawing
Flow chart and block diagram show the device of multiple embodiments according to the present invention, the possibility of method and computer program product is realized
Architectural framework, function and operation.In this regard, each box in flow chart or block diagram can represent module, a program
A part for section or code, a part for the module, program segment or code are used to implement defined patrol comprising one or more
Collect the executable instruction of function.It should also be noted that at some as the function of in the realization method replaced, being marked in box
It can be occurred with being different from the sequence marked in attached drawing.For example, two continuous boxes can essentially be held substantially in parallel
Row, they can also be performed in the opposite order sometimes, this is depended on the functions involved.It is also noted that block diagram and/or
The combination of each box in flow chart and the box in block diagram and/or flow chart can use function or dynamic as defined in performing
The dedicated hardware based system made is realized or can be realized with the combination of specialized hardware and computer instruction.
In addition, each function module in each embodiment of the present invention can integrate to form an independent portion
Point or modules individualism, can also two or more modules be integrated to form an independent part.
If the function is realized in the form of software function module and is independent product sale or in use, can be with
It is stored in a computer read/write memory medium.Based on such understanding, technical scheme of the present invention is substantially in other words
The part contribute to the prior art or the part of the technical solution can be embodied in the form of software product, the meter
Calculation machine software product is stored in a storage medium, is used including some instructions so that a computer equipment (can be
People's computer, electronic equipment or network equipment etc.) perform all or part of step of each embodiment the method for the present invention
Suddenly.And aforementioned storage medium includes：USB flash disk, read-only memory (ROM, Read-Only Memory), is deposited mobile hard disk at random
The various media that can store program code such as access to memory (RAM, Random Access Memory), magnetic disc or CD.
It should be noted that herein, term " comprising ", "comprising" or its any other variant are intended to the packet of nonexcludability
Contain so that process, method, article or equipment including a series of elements not only include those elements, but also including
It other elements that are not explicitly listed or further includes as elements inherent to such a process, method, article, or device.
In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including the element
Process, method, also there are other identical elements in article or equipment.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, that is made any repaiies
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (10)
1. a kind of log collecting method, which is characterized in that the method includes：
The log information that journal file generates is obtained, the log information is packaged to obtain log event, and by described in
Log event is sent to corresponding output module, wherein, different journal files is corresponding with different output modules, and the day
Will event includes routing information；
Judge connection container in whether there is with the matched log path of the routing information, wherein, it is described connection container in wrap
Include the connection between multiple log paths and the multiple log path and corresponding message queue Categories Address；
When connecting in container in the presence of log path matched with the routing information, pass through the log path and message queue point
Log event in the output module is sent to the message queue Categories Address by the connection between class address.
2. log collecting method according to claim 1, which is characterized in that be not present and the path when connecting in container
During the log path of information matches, the method further includes：
Log path corresponding with the routing information and the corresponding message team of the log path are searched from preset database
Row Categories Address creates the connection between the log path and the corresponding message queue Categories Address and preserves to the connection
In container, wherein, the correspondence being stored in the database between different log paths and message queue Categories Address；
It will be in the output module by the connection between the log path of establishment and corresponding message queue Categories Address
Log event be sent to the message queue Categories Address.
3. log collecting method according to claim 1, which is characterized in that in the daily record for performing acquisition journal file generation
Information and the step of be packaged to obtain log event before, the method further includes：
The output module and corresponding instantiation procedure are created in log collection tool and is preserved into data dictionary,
In, the corresponding instantiation procedure of different output modules is different；
Search the title of corresponding output module from the data dictionary according to the corresponding module title of journal file, and according to
The corresponding instantiation procedure of title and journal file of output module create connection container.
4. log collecting method according to claim 1, which is characterized in that be sent to the log event corresponding defeated
The step of going out module includes：
Call the interface method of the corresponding output module of the log event that the log event is sent to corresponding output mould
Block.
5. log collecting method according to claim 1, which is characterized in that the output module is file output processing mould
Block or database output processing module.
6. a kind of log collection device, which is characterized in that described device includes：
Log processing module for obtaining the log information of journal file generation, the log information is packaged to obtain
Log event, and the log event is sent to corresponding output module, wherein, different journal files is corresponding with different
Output module, and the log event includes routing information；
Judgment module, for judge connect container in whether there is with the matched log path of the routing information, wherein, it is described
Connection container is included between multiple log paths and the multiple log path and corresponding message queue Categories Address
Connection；
Sending module, for when there is log path matched with the routing information in connecting container, passing through the daily record road
Log event in the output module is sent to message queue classification by the connection between diameter and message queue Categories Address
Address.
7. log collection device according to claim 6, which is characterized in that be not present and the path when connecting in container
During the log path of information matches, described device further includes：
Connection establishment module, for searching log path corresponding with the routing information and the daily record from preset database
The corresponding message queue Categories Address in path creates the connection between the log path and the corresponding message queue Categories Address
And preserve into the connection container, wherein, different log paths and message queue are stored in the database taxonomically
Correspondence between location；
The sending module is additionally operable to through the company between the log path of establishment and corresponding message queue Categories Address
It connects and the log event in the output module is sent to the message queue Categories Address.
8. log collection device according to claim 6, which is characterized in that described device further includes：
First creation module, for creating the output module and corresponding instantiation procedure in log collection tool and preserving
Into data dictionary, wherein, the corresponding instantiation procedure of different output modules is different；
Second creation module, for searching corresponding output from the data dictionary according to the corresponding module title of journal file
The title of module, and connection container is created according to the corresponding instantiation procedure of title and journal file of output module.
9. log collection device according to claim 6, which is characterized in that the log processing module is additionally operable to call
The log event is sent to corresponding output module by the interface method of the corresponding output module of the log event.
10. log collection device according to claim 6, which is characterized in that the output module is handled for file output
Module or database output processing module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810015655.4A CN108170578B (en) | 2018-01-08 | 2018-01-08 | Log collection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810015655.4A CN108170578B (en) | 2018-01-08 | 2018-01-08 | Log collection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108170578A true CN108170578A (en) | 2018-06-15 |
| CN108170578B CN108170578B (en) | 2021-05-28 |
Family
ID=62517589
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810015655.4A Active CN108170578B (en) | 2018-01-08 | 2018-01-08 | Log collection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108170578B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109034657A (en) * | 2018-08-22 | 2018-12-18 | 泰康保险集团股份有限公司 | Process path finding method, device, medium and electronic equipment based on block chain |
| CN109144965A (en) * | 2018-08-30 | 2019-01-04 | 百度在线网络技术（北京）有限公司 | A kind of log processing method, device, equipment and medium |
| CN111008180A (en) * | 2019-10-29 | 2020-04-14 | 烽火通信科技股份有限公司 | Method and device for collecting log files in container and electronic equipment |
| CN112306979A (en) * | 2020-10-30 | 2021-02-02 | 山东浪潮通软信息科技有限公司 | Message queue-based log information processing method and device and readable medium |
| CN112364285A (en) * | 2020-11-23 | 2021-02-12 | 北京八分量信息科技有限公司 | Method and device for establishing anomaly detection model based on UEBA (unified extensible architecture), and related product |
| CN112364286A (en) * | 2020-11-23 | 2021-02-12 | 北京八分量信息科技有限公司 | Method and device for abnormality detection based on UEBA and related product |
| CN112468359A (en) * | 2020-11-10 | 2021-03-09 | 许继集团有限公司 | Log acquisition system and method for power system Internet of things edge Internet of things agent device |
| CN113420003A (en) * | 2021-06-30 | 2021-09-21 | 中国航空油料有限责任公司 | Method, device, equipment and medium for processing data interaction log |
| CN114915543A (en) * | 2022-05-07 | 2022-08-16 | 中国农业银行股份有限公司 | Message queue monitoring method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006130509A1 (en) * | 2005-05-27 | 2006-12-07 | Computer Associates Think, Inc. | Method and system for visualizing job forecasting in a computer system |
| CN102385549A (en) * | 2010-09-02 | 2012-03-21 | 北京无限立通通讯技术有限责任公司 | Log processing system, log processing method and log storage sub-system |
| CN103617283A (en) * | 2013-12-11 | 2014-03-05 | 北京京东尚科信息技术有限公司 | Method and device for storing logs |
| CN104375928A (en) * | 2013-08-12 | 2015-02-25 | 鸿富锦精密工业（深圳）有限公司 | Abnormal log management method and system |
| CN105991633A (en) * | 2015-04-28 | 2016-10-05 | 杭州迪普科技有限公司 | Session log processing method and device |
-
2018
- 2018-01-08 CN CN201810015655.4A patent/CN108170578B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006130509A1 (en) * | 2005-05-27 | 2006-12-07 | Computer Associates Think, Inc. | Method and system for visualizing job forecasting in a computer system |
| CN102385549A (en) * | 2010-09-02 | 2012-03-21 | 北京无限立通通讯技术有限责任公司 | Log processing system, log processing method and log storage sub-system |
| CN104375928A (en) * | 2013-08-12 | 2015-02-25 | 鸿富锦精密工业（深圳）有限公司 | Abnormal log management method and system |
| CN103617283A (en) * | 2013-12-11 | 2014-03-05 | 北京京东尚科信息技术有限公司 | Method and device for storing logs |
| CN105991633A (en) * | 2015-04-28 | 2016-10-05 | 杭州迪普科技有限公司 | Session log processing method and device |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109034657A (en) * | 2018-08-22 | 2018-12-18 | 泰康保险集团股份有限公司 | Process path finding method, device, medium and electronic equipment based on block chain |
| CN109144965A (en) * | 2018-08-30 | 2019-01-04 | 百度在线网络技术（北京）有限公司 | A kind of log processing method, device, equipment and medium |
| CN111008180A (en) * | 2019-10-29 | 2020-04-14 | 烽火通信科技股份有限公司 | Method and device for collecting log files in container and electronic equipment |
| CN112306979A (en) * | 2020-10-30 | 2021-02-02 | 山东浪潮通软信息科技有限公司 | Message queue-based log information processing method and device and readable medium |
| CN112306979B (en) * | 2020-10-30 | 2022-11-01 | 浪潮通用软件有限公司 | Message queue-based log information processing method and device and readable medium |
| CN112468359A (en) * | 2020-11-10 | 2021-03-09 | 许继集团有限公司 | Log acquisition system and method for power system Internet of things edge Internet of things agent device |
| CN112364285A (en) * | 2020-11-23 | 2021-02-12 | 北京八分量信息科技有限公司 | Method and device for establishing anomaly detection model based on UEBA (unified extensible architecture), and related product |
| CN112364286A (en) * | 2020-11-23 | 2021-02-12 | 北京八分量信息科技有限公司 | Method and device for abnormality detection based on UEBA and related product |
| CN112364285B (en) * | 2020-11-23 | 2024-02-02 | 北京八分量信息科技有限公司 | Method and device for establishing abnormality detection model based on UEBA (unified extensible firmware interface) and related products |
| CN113420003A (en) * | 2021-06-30 | 2021-09-21 | 中国航空油料有限责任公司 | Method, device, equipment and medium for processing data interaction log |
| CN114915543A (en) * | 2022-05-07 | 2022-08-16 | 中国农业银行股份有限公司 | Message queue monitoring method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108170578B (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108170578A (en) | log collecting method and device | |
| CN105427129B (en) | Information delivery method and system | |
| JP6148732B2 (en) | Data indexing method and apparatus | |
| CN102902775B (en) | The method and system that internet calculates in real time | |
| CN112559809B (en) | Consumer multi-channel data integration method, system, device and storage medium | |
| CN106844143A (en) | A kind of daily record duplicate removal treatment method and device | |
| CN107341220A (en) | A kind of multi-source data fusion method and device | |
| WO2015078303A1 (en) | Methods and devices for managing emails | |
| CN108510096A (en) | Trade company's attrition prediction method, apparatus, equipment and storage medium | |
| CN108319598A (en) | data cache method, device and system | |
| CN110689084B (en) | Abnormal user identification method and device | |
| CN108090203A (en) | Video classification methods, device, storage medium and electronic equipment | |
| CN113643062A (en) | Advertisement display method of novel aggregation advertisement system | |
| CN108132868A (en) | A kind of data monitoring method, device, computing device and storage medium | |
| CN110708285A (en) | Flow monitoring method, device, medium and electronic equipment | |
| CN108846098A (en) | A kind of information flow summarization generation and methods of exhibiting | |
| CN110266603B (en) | System and method for analyzing network flow of identity authentication service based on HTTP (hyper text transport protocol) | |
| CN106304085B (en) | Information processing method and device | |
| CN105337848B (en) | The remark names preparation method and device of network good friend | |
| CN107888419A (en) | A kind of switch network Topology g eneration method and device | |
| CN105095343A (en) | Information processing method, information display method, information processing device and information display device | |
| CN112910890B (en) | Anonymous network flow fingerprint identification method and device based on time convolution network | |
| Sharma et al. | Comparative analysis of different algorithms in link prediction on social networks | |
| CN108255868A (en) | Check the method and apparatus linked in website | |
| CN108536855A (en) | Mobile communications device evidence collecting method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240627 Address after: Room 218, No. 14 Longlin West Street, Tianhe District, Guangzhou City, Guangdong Province, 510000 Patentee after: Guangzhou Haoruitai Information Technology Co.,Ltd. Country or region after: China Address before: 11 / F, building B1, software industry phase 4.1, No.1, Software Park East Road, Donghu New Technology Development Zone, Wuhan City, Hubei Province, 430000 Patentee before: WUHAN DOUYU NETWORK TECHNOLOGY Co.,Ltd. Country or region before: China |