CN106161066A - A kind of log collection method and server - Google Patents
A kind of log collection method and server Download PDFInfo
- Publication number
- CN106161066A CN106161066A CN201510171909.8A CN201510171909A CN106161066A CN 106161066 A CN106161066 A CN 106161066A CN 201510171909 A CN201510171909 A CN 201510171909A CN 106161066 A CN106161066 A CN 106161066A
- Authority
- CN
- China
- Prior art keywords
- daily record
- transaction message
- message daily
- server
- reliable
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a kind of log collection method and server, obtain journal file；It is collected described journal file processing, it is thus achieved that transaction message daily record；Described transaction message daily record is distributed to reliable and secure queue.
Description
Technical field
The present invention relates to data processing technique, particularly relate to a kind of log collection method and server.
Background technology
Along with the development of business, in order to obtain the data message of user's perception in time and based on acquired
Data message carries out the fault diagnosis of necessity, generally uses traditional log collection technology to obtain data message.
Specifically, disposing multiple the same capture program, a capture program can gather on multiple host simultaneously
Log content, collects all of log content afterwards and is stored in disk with the form of journal file.So,
Needs carry out log content check time, need to be opened manually journal file and entered by the command information searched
Line search judges whether the log content every time searched belongs to log content to be found.
But, there is following technical problem in traditional log collection technology:
1) daily record is directly gathered, it is impossible to log content is compressed, takies network bandwidth resources big；
2) cannot give for change after packet drop, and packet loss easily occurring；
3) log collection of magnanimity, causes log collection time delay serious, it is impossible to realize the Real-time Collection of daily record.
Summary of the invention
In view of this, embodiment of the present invention expectation provides a kind of log collection method and server, it is possible to protecting
In the case of card saves network bandwidth resources and log content integrity, it is achieved the Real-time Collection to log content.
For reaching above-mentioned purpose, the technical scheme of the embodiment of the present invention is achieved in that
The embodiment of the present invention provides a kind of log collection method, and described method includes:
Obtain journal file；
It is collected described journal file processing, it is thus achieved that transaction message daily record；
Described transaction message daily record is distributed to reliable and secure queue.
In such scheme, described be collected described journal file processes, it is thus achieved that transaction message daily record, bag
Include:
According to dynamic link library API, described journal file is carried out assembling process, to form transaction message daily record.
In such scheme, described described transaction message daily record is distributed to reliable and secure queue, including:
Corresponding pass between serial number and reliable and secure queue according to described transaction message daily record correspondence message
System, stores described transaction message daily record distribution to corresponding reliable and secure queue.
In such scheme, described method also includes:
In the case of transaction log switch is opened or network is obstructed, by described transaction log file record at this
In ground file.
In such scheme, described method also includes:
Described transaction message daily record is carried out abnormal verification；
When determining that described transaction message daily record exists abnormal conditions, please by initiating detailed transaction log collection
The mode asked reacquires transaction message daily record.
The embodiment of the present invention also provides for a kind of server, and described server includes that acquisition module, collection process mould
Block and distribution module；
Described acquisition module, is used for obtaining journal file；
Described collection processing module, for being collected process, it is thus achieved that transaction message day to described journal file
Will；
Described distribution module, for being distributed to reliable and secure queue by described transaction message daily record.
In such scheme, described collection processing module, specifically for according to dynamic link library API to described day
Will file carries out assembling process, to form transaction message daily record.
In such scheme, described distribution module, specifically for according to described transaction message daily record correspondence message
Corresponding relation between serial number and reliable and secure queue, stores described transaction message daily record distribution to correspondence
Reliable and secure queue.
In such scheme, described server also includes logging modle；
Described logging modle, in the case of transaction log switch is opened or network is obstructed, by described friendship
Easily journal file record is in local file.
In such scheme, described server also includes correction verification module；
Described correction verification module, for carrying out abnormal verification to described transaction message daily record；
Described acquisition module, is additionally operable to when determining that described transaction message daily record exists abnormal conditions, by sending out
The mode playing detailed transaction log collection request reacquires transaction message daily record.
Log collection method that the embodiment of the present invention is provided and server, obtain journal file；To described day
Will file is collected processing, it is thus achieved that transaction message daily record；It is distributed to reliably pacify by described transaction message daily record
Whole team arranges.So, it is possible in the case of ensureing to save network bandwidth resources and log content integrity, real
The now Real-time Collection to log content.
Accompanying drawing explanation
Fig. 1 be embodiment of the present invention log collection method realize schematic flow sheet one；
Fig. 2 be embodiment of the present invention log collection method realize schematic flow sheet two；
Fig. 3 be embodiment of the present invention log collection method realize schematic flow sheet three；
Fig. 4 is the composition structural representation one of embodiment of the present invention server；
Fig. 5 is the composition structural representation two of embodiment of the present invention server；
Fig. 6 is the composition structural representation three of embodiment of the present invention server；
Fig. 7 be the present invention one application example log collection method implement schematic flow sheet.
Detailed description of the invention
In embodiments of the present invention, journal file is obtained；It is collected described journal file processing, it is thus achieved that
Transaction message daily record；Described transaction message daily record is distributed to reliable and secure queue.
Below in conjunction with the accompanying drawings and specific embodiment the present invention is further described in more detail.
Embodiment one
Fig. 1 be embodiment of the present invention log collection method realize schematic flow sheet one, as it is shown in figure 1, this
Inventive embodiments log collection method includes:
Step S101: obtain journal file；
Specifically, the day generated by daily record generation server for realizing the server of log collection function to obtain
Will file；The mode of described acquisition journal file can be to obtain in real time, it is also possible to is periodically to obtain.When
The mode of described acquisition journal file is selected when obtaining in real time, can ensure the real-time of log collection well,
But this acquisition mode is to sacrifice network bandwidth resources as cost；Side when described acquisition journal file
Formula is selected when periodically obtaining, and needs when determining the size in cycle, the real-time of efficient balance log collection
Expense with network bandwidth resources.Described server can include smart mobile phone, panel computer, notebook electricity
Brain and personal digital assistant etc..
Step S102: be collected described journal file processing, it is thus achieved that transaction message daily record；
Specifically, the server realizing log collection function it is used for according to dynamic link library API to described daily record
File carries out assembling process, to form transaction message daily record.Owing to being the most provided with API in described API
Function, therefore after server gets journal file, can be entered described journal file by api function
Row assembling processes.
Step S103: described transaction message daily record is distributed to reliable and secure queue.
Specifically, server is according to the serial number of described transaction message daily record correspondence message and reliable and secure queue
Between corresponding relation, described transaction message daily record distribution is stored to corresponding reliable and secure queue.Here,
API in server is first according to serial number and the reliable and secure queue of described transaction message daily record correspondence message
Sum carry out simple modulo operation, to ensure to have the message of identical serial number, enter identical reliable
Safe queue.
So, by embodiment of the present invention log collection method, it is possible to ensure save network bandwidth resources and
In the case of log content integrity, it is achieved the Real-time Collection to log content.
Embodiment two
Fig. 2 be embodiment of the present invention log collection method realize schematic flow sheet two, as in figure 2 it is shown, this
Inventive embodiments log collection method includes:
Step S101: obtain journal file；
Step S102: be collected described journal file processing, it is thus achieved that transaction message daily record；
Step S103: described transaction message daily record is distributed to reliable and secure queue；
Step S101 in the embodiment of the present invention two can respectively correspondingly see embodiment one to step S103
In step S101 to step S103, for saving length, repeat no more here.
Step S201: in the case of transaction log switch is opened or network is obstructed, by described transaction log literary composition
Part record is in local file.
Specifically, server during described transaction message daily record is distributed to reliable and secure queue, when
There is transaction log switch to open or time network is obstructed, by described transaction log file record in local file.
So, by embodiment of the present invention log collection method, it is possible to ensure save network bandwidth resources and
In the case of log content integrity, it is achieved the Real-time Collection to log content.
Embodiment three
Fig. 3 be embodiment of the present invention log collection method realize schematic flow sheet three, as it is shown on figure 3, this
Inventive embodiments log collection method includes:
Step S101: obtain journal file；
Step S102: be collected described journal file processing, it is thus achieved that transaction message daily record；
Step S103: described transaction message daily record is distributed to reliable and secure queue；
Step S101 in the embodiment of the present invention three can respectively correspondingly see embodiment one to step S103
In step S101 to step S103, for saving length, repeat no more here.
Step S301: described transaction message daily record is carried out abnormal verification；
Specifically, the server the realizing log collection function transaction to storing to reliable and secure queue it is used for
Message daily record carries out abnormal verification, to determine whether described transaction message daily record exists exception, such as described transaction
Whether message daily record itself exists process of exchange corresponding to abnormal or described transaction message daily record and belongs to spy automatically
Survey.
Step S302: when determining that described transaction message daily record exists abnormal conditions, by initiating detail transaction
The mode of log collection request reacquires transaction message daily record.
Specifically, when by abnormality detection, determining when described transaction message daily record exists abnormal conditions, be used for
The server realizing log collection function reacquires friendship by the way of initiating detailed transaction log collection request
Easily message daily record.
So, by embodiment of the present invention log collection method, it is possible to ensure save network bandwidth resources and
In the case of log content integrity, it is achieved the Real-time Collection to log content.
Embodiment four
Fig. 4 is the composition structural representation of embodiment of the present invention server, as shown in Figure 4, described server
Including acquisition module 401, collect processing module 402 and distribution module 403；
Described acquisition module 401, is used for obtaining journal file；
Described collection processing module 402, for being collected process to described journal file, it is thus achieved that transaction report
Literary composition daily record；
Specifically, described journal file is carried out by described collection processing module 402 according to dynamic link library API
Assembling processes, to form transaction message daily record.
Described distribution module 403, for being distributed to reliable and secure queue by described transaction message daily record.
Specifically, described distribution module 403 according to the serial number of described transaction message daily record correspondence message with can
By the corresponding relation between safe queue, described transaction message daily record distribution is stored to corresponding reliable and secure
Queue.
In one embodiment, as it is shown in figure 5, described server also includes logging modle 404；
Described logging modle 404, in the case of transaction log switch is opened or network is obstructed, by institute
State transaction log file record in local file.
In one embodiment, as shown in Figure 6, described server also includes correction verification module 405；
Described correction verification module 405, for carrying out abnormal verification to described transaction message daily record；
Described acquisition module 401, is additionally operable to when determining that described transaction message daily record exists abnormal conditions, logical
The mode crossing initiation detailed transaction log collection request reacquires transaction message daily record.
In actual applications, each module in server described in the embodiment of the present invention all can be by device
Processor realizes, it is also possible to realized by concrete logic circuit；Such as, in actual applications, can be by position
Central processing unit (CPU) in described server, microprocessor (MPU), digital signal processor
(DSP) or field programmable gate array (FPGA) realize.
Concrete application scenarios:
Fig. 7 be the present invention one application example log collection method implement schematic flow sheet, such as Fig. 7 institute
Showing, described log collection method specifically includes:
Step 1, needs generate server disposition log concentrator program in daily record；
Step 2, log concentrator connect reliable and secure queue, for the connection of each reliable and secure queue,
Setting up buffer queue, the data in described buffer queue mail to reliable and secure queue by merging after being compressed；
Wherein, described reliable and secure queue has multiple.
Step 3, log concentrator are after receiving the message that API sends over, according to entering that API distributes
Which reliable and secure queue, is saved in corresponding buffer queue by transaction message daily record；
Step 4, the buffer queue of log concentrator need to limit size, when message belonging to transaction message daily record
When sending abnormal, if the size of buffer queue exceedes the size of restriction, then by the report in whole buffer queue
Local file write in literary composition；The most just turn back to buffer queue by sending failed message.
So, in application example of the present invention, by API log concentrator, log content is compressed；
When message belonging to transaction message daily record is being sent to reliable and secure queue by log concentrator, work as friendship
Easily daily record switch open or network obstructed in the case of, by described transaction log file record in local file.
So, it is possible in the case of ensureing to save network bandwidth resources and log content integrity, it is achieved to daily record
The Real-time Collection of content.
The above, only presently preferred embodiments of the present invention, it is not intended to limit the protection model of the present invention
Enclose.
Claims (10)
1. a log collection method, it is characterised in that described method includes:
Obtain journal file；
It is collected described journal file processing, it is thus achieved that transaction message daily record；
Described transaction message daily record is distributed to reliable and secure queue.
Method the most according to claim 1, it is characterised in that described described journal file is received
Collection processes, it is thus achieved that transaction message daily record, including:
According to dynamic link library API, described journal file is carried out assembling process, to form transaction message daily record.
3. according to the method described in right 1, it is characterised in that described described transaction message daily record is distributed to
Reliable and secure queue, including:
Corresponding pass between serial number and reliable and secure queue according to described transaction message daily record correspondence message
System, stores described transaction message daily record distribution to corresponding reliable and secure queue.
4. according to the method described in any one of claims 1 to 3, it is characterised in that described method also includes:
In the case of transaction log switch is opened or network is obstructed, by described transaction log file record at this
In ground file.
5. according to the method described in any one of claims 1 to 3, it is characterised in that described method also includes:
Described transaction message daily record is carried out abnormal verification；
When determining that described transaction message daily record exists abnormal conditions, please by initiating detailed transaction log collection
The mode asked reacquires transaction message daily record.
6. a server, it is characterised in that described server include acquisition module, collect processing module and
Distribution module；
Described acquisition module, is used for obtaining journal file；
Described collection processing module, for being collected process, it is thus achieved that transaction message day to described journal file
Will；
Described distribution module, for being distributed to reliable and secure queue by described transaction message daily record.
Server the most according to claim 6, it is characterised in that
Described collection processing module, specifically for carrying out group according to dynamic link library API to described journal file
Dress processes, to form transaction message daily record.
8. according to the server described in right 6, it is characterised in that
Described distribution module, specifically for the serial number according to described transaction message daily record correspondence message with reliable
Corresponding relation between safe queue, stores described transaction message daily record distribution to corresponding reliable and secure team
Row.
9. according to the server described in any one of claim 6 to 8, it is characterised in that described server is also
Including logging modle；
Described logging modle, in the case of transaction log switch is opened or network is obstructed, by described friendship
Easily journal file record is in local file.
10. according to the server described in any one of claim 6 to 8, it is characterised in that described server
Also include correction verification module；
Described correction verification module, for carrying out abnormal verification to described transaction message daily record；
Described acquisition module, is additionally operable to when determining that described transaction message daily record exists abnormal conditions, by sending out
The mode playing detailed transaction log collection request reacquires transaction message daily record.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510171909.8A CN106161066A (en) | 2015-04-13 | 2015-04-13 | A kind of log collection method and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510171909.8A CN106161066A (en) | 2015-04-13 | 2015-04-13 | A kind of log collection method and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106161066A true CN106161066A (en) | 2016-11-23 |
Family
ID=57335934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510171909.8A Pending CN106161066A (en) | 2015-04-13 | 2015-04-13 | A kind of log collection method and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106161066A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108011966A (en) * | 2017-12-14 | 2018-05-08 | 广东金赋科技股份有限公司 | The optimization method that a kind of self-aided terminal log compression uploads |
| CN111798237A (en) * | 2020-06-30 | 2020-10-20 | 中国工商银行股份有限公司 | Abnormal transaction diagnosis method and system based on application log |
| CN111935140A (en) * | 2020-08-10 | 2020-11-13 | 中国工商银行股份有限公司 | Abnormal message identification method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101668155A (en) * | 2009-09-10 | 2010-03-10 | 中兴通讯股份有限公司 | IPTV system log message tracing method and system |
| CN102461118A (en) * | 2009-06-11 | 2012-05-16 | 松下航空电子公司 | System and method for providing security aboard a moving platform |
| CN102460441A (en) * | 2009-06-15 | 2012-05-16 | 弗里塞恩公司 | Method and system for auditing transaction data from database operations |
| CN102902764A (en) * | 2012-09-25 | 2013-01-30 | 北京奇虎科技有限公司 | Method and device for log recording |
-
2015
- 2015-04-13 CN CN201510171909.8A patent/CN106161066A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102461118A (en) * | 2009-06-11 | 2012-05-16 | 松下航空电子公司 | System and method for providing security aboard a moving platform |
| CN102460441A (en) * | 2009-06-15 | 2012-05-16 | 弗里塞恩公司 | Method and system for auditing transaction data from database operations |
| CN101668155A (en) * | 2009-09-10 | 2010-03-10 | 中兴通讯股份有限公司 | IPTV system log message tracing method and system |
| CN102902764A (en) * | 2012-09-25 | 2013-01-30 | 北京奇虎科技有限公司 | Method and device for log recording |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108011966A (en) * | 2017-12-14 | 2018-05-08 | 广东金赋科技股份有限公司 | The optimization method that a kind of self-aided terminal log compression uploads |
| CN111798237A (en) * | 2020-06-30 | 2020-10-20 | 中国工商银行股份有限公司 | Abnormal transaction diagnosis method and system based on application log |
| CN111798237B (en) * | 2020-06-30 | 2023-08-25 | 中国工商银行股份有限公司 | Abnormal transaction diagnosis method and system based on application log |
| CN111935140A (en) * | 2020-08-10 | 2020-11-13 | 中国工商银行股份有限公司 | Abnormal message identification method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112313916B (en) | Method and system for pseudo-storage of anti-tampering logs by fusing block chain technology | |
| CN102937932B (en) | The automatic test device of integrated isomery testing tool and method | |
| US20040003327A1 (en) | Method and system to implement a system event log for system manageability | |
| CN103401698B (en) | For the monitoring system that server health is reported to the police in server set group operatione | |
| CN106330963A (en) | Cross-network multi-node log collecting method | |
| CN106161066A (en) | A kind of log collection method and server | |
| CN107249027A (en) | A kind of intelligent device management system based on internet | |
| CN107800663A (en) | The detection method and device of flow off-line files | |
| CN107819616A (en) | Automatically extract the method, apparatus and system of daily record | |
| CN101388036A (en) | Data table summarizing method and device | |
| US9396145B1 (en) | In-chip bus tracer | |
| CN103077080B (en) | Based on parallel program performance collecting method and the device of high performance platform | |
| CN108389131A (en) | A kind of financial system micro services transaction monitoring implementation method | |
| CN107391374A (en) | Middleware automatic checking method | |
| CN103024023A (en) | Electronic reading room network auditing method | |
| CN105468502A (en) | Log collection method, device and system | |
| CN104598202A (en) | Command line management device and method | |
| CN107463340A (en) | The data-storage system of computer | |
| CN201663613U (en) | Novel application integration gateway system | |
| CN103701595B (en) | System, method and apparatus for login authentication | |
| CN106304122A (en) | A kind of business datum analyzes method and system | |
| CN105930299B (en) | BFM-based SystemVerilog protocol verification platform building method | |
| CN108985083A (en) | A kind of Computer Data Security management system and method | |
| CN102624567A (en) | Hardware type network safety control server thermal backup device | |
| CN104426708A (en) | Method and system for executing security detection service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161123 |
|
| RJ01 | Rejection of invention patent application after publication |