CN103856534B - A kind of daily record gathering method and system based on peer-to-peer network - Google Patents
A kind of daily record gathering method and system based on peer-to-peer network Download PDFInfo
- Publication number
- CN103856534B CN103856534B CN201210516908.9A CN201210516908A CN103856534B CN 103856534 B CN103856534 B CN 103856534B CN 201210516908 A CN201210516908 A CN 201210516908A CN 103856534 B CN103856534 B CN 103856534B
- Authority
- CN
- China
- Prior art keywords
- peer
- system server
- rpc message
- log
- log system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Debugging And Monitoring (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kind of daily record gathering method and system based on peer-to-peer network, including：After current log system server receives the remote procedure call protocol RPC message that corresponding business module server is sent, according to the state of itself and the state of reciprocity log system server, RPC message is handled, including：By the log information recording in RPC message into file, with as journal file, or, RPC message is transmitted to reciprocity log system server；Each log system server in peer-to-peer network uploads to each self-recording journal file the document storage server specified.Service logic is realized by the present invention and record logic is uncoupled while improve the Performance And Reliability of blog search system.
Description
Technical field
The present invention relates to file search technical field, more particularly to a kind of daily record gathering method based on peer-to-peer network and it is
System.
Background technology
At present, daily record gathering system is broadly divided into two major classes：One kind is traditional star Nonseparation daily record gathering system, such as
Shown in Fig. 1, log information is produced (such as the request response server of service) by business module server in the system and is recorded as
Journal file, then some or some addresses (ftp file server or NFS) are uploaded to, collect journal file.One
Kind is the separate type daily record gathering system based on hierarchical network, and the typical of such system is represented as what Facebook increased income
Scribe systems, this kind of system mainly have following characteristics：1st, log system server is clothes for business module server
Business device, remote procedure call protocol (RPC, Remote the Procedure Call that module server of accepting business is sent
Protocol) ask.The log information of generation is transmitted to log system server by business module server by RPC modes.2、
Log system server can be journal file the log information recording received, log information can also be sent to next stage
Log system server, the processing procedure of daily record is based on configuration.
Above-mentioned daily record gathering system is primarily present the problem of following：
In traditional star Nonseparation daily record gathering system, business module server produces log information and by log information
File is recorded, on the same server, this mode exists as follows for the interrelated logic and service logic of such log recording
Shortcoming：1st, the service logic logic related to log recording intercouples, and performance can only take both low values.Log recording is general
A large amount of IO consumption can be taken, business service performance can be influenceed.2nd, another benefit that log recording separates with business is：Daily record is remembered
Record more flexible.Service logic need not pay close attention to the related logic of log recording and (such as journal format, recorded local or network text
Part system etc.), and traditional star Nonseparation daily record gathering system does not possess these features really, comparatively log recording is ineffective
Living, service logic needs to pay close attention to the related logic of log recording, can influence business service performance.3rd, on from log recording to daily record
Biography is all single line logic, i.e. some link error can all cause log recording unsuccessful, without reliable fault tolerant mechanism.
Separate type daily record gathering system based on hierarchical network, has a disadvantage that：The network structure of such system is single, day
Will record efficiency and reliability are low.
The content of the invention
In view of this, it is a primary object of the present invention to provide a kind of daily record gathering method based on peer-to-peer network and be
System, to solve the problems, such as that log recording efficiency and reliability is low in existing daily record gathering system, influence business service performance.
To reach above-mentioned purpose, the technical proposal of the invention is realized in this way：
The invention provides a kind of daily record gathering method based on peer-to-peer network, establish by multiple log system server groups
Into peer-to-peer network, this method also includes：
Current log system server receives the remote procedure call protocol RPC that corresponding business module server is sent
After message, according to the state of itself and the state of reciprocity log system server, the RPC message is handled, including：Will
Log information recording in the RPC message is into file, with as journal file, or, the RPC message is transmitted to pair
Deng log system server；
Each log system server in the peer-to-peer network uploads to each self-recording journal file the text specified
Part storage server.
Preferably, the state of log system server includes：React type, ability and the place net of log system server
Between the ability information of network, the performance information for reacting log system server load and reaction and reciprocity log system server
The range information of distance.
Preferably, it is described to include log information recording to file：By log information recording to local file or network
In file.
Preferably, will after the current log system server or reciprocity log system server receive RPC message
RPC message is put into the task queue of itself.
Preferably, according to following rule when handling the RPC message：
According to the RPC message in the sequential processes task queue of priority from high to low；And/or in statistics task queue
The type of RPC message, RPC message corresponding to the most type of priority treatment message number；And/or for same type of RPC
Message, priority treatment enter the earliest RPC message of task queue.
Present invention also offers a kind of daily record gathering system based on peer-to-peer network, including：
Business module server, corresponding log system service is sent to for producing log information, and by RPC message
Device；
Log system server, after the RPC message sent for business module server corresponding to receiving, according to itself
State and reciprocity log system server state, the RPC message is handled, including：By in the RPC message
Log information recording is into file, with as journal file, or, the RPC message is transmitted to the reciprocity log system
Server；
Log system server, it is additionally operable to uploading to the journal file of self record into the document storage server specified.
Preferably, the state of log system server includes：React type, ability and the place net of log system server
Between the ability information of network, the performance information for reacting log system server load and reaction and reciprocity log system server
The range information of distance.
Preferably, the log system server, it is additionally operable in log information recording to local file or network file.
Preferably, the log system server or the reciprocity log system server, the RPC for being additionally operable to receive disappear
It is put into after breath in the task queue of itself.
Preferably, the log system server, it is additionally operable to handle RPC message according to following rule：According to excellent
RPC message in the sequential processes task queue of first level from high to low；And/or in statistics task queue RPC message type,
RPC message corresponding to the most type of priority treatment message number；And/or for same type of RPC message, priority treatment
The RPC message earliest into task queue.
The method and system that daily record of the present invention based on peer-to-peer network is collected：Initially set up by multiple log system servers
The peer-to-peer network of composition, in a peer-to-peer network, what business module server corresponding to the reception of current log system server was sent
RPC message, and according to itself state and the state of reciprocity log system server, RPC message is handled, including：Will
Log information recording in RPC message is into file, with as journal file, or, RPC message is transmitted to reciprocity day aspiration
System server.With it, the interrelated logic of log recording is performed by log system server, service logic is by business module
Server performs, and so, service logic and record logic are decoupling, improve the performance of business module server, meanwhile, utilize
The high efficiency and reliability of peer-to-peer network, have also greatly improved the Performance And Reliability of blog search system.
Brief description of the drawings
Fig. 1 is traditional star Nonseparation daily record gathering system schematic diagram；
Fig. 2 is the separate type daily record gathering system schematic diagram based on hierarchical network；
Fig. 3 is the daily record gathering system schematic diagram of the invention based on peer-to-peer network.
Embodiment
The technical solution of the present invention is further elaborated with specific embodiment below in conjunction with the accompanying drawings.
The basic thought that daily record of the present invention based on peer-to-peer network is collected is：Initially set up by multiple log system servers
The peer-to-peer network of composition, in a peer-to-peer network, what business module server corresponding to the reception of current log system server was sent
RPC message, and according to itself state and the state of reciprocity log system server, RPC message is handled, including：Will
Log information recording in RPC message is into file, with as journal file, or, RPC message is transmitted to reciprocity day aspiration
System server.With it, the interrelated logic of log recording is performed by log system server, service logic is by business module
Server performs, and so, service logic and record logic are decoupling, improve the performance of business module server, meanwhile, utilize
The high efficiency and reliability of peer-to-peer network, have also greatly improved the Performance And Reliability of blog search system.
Daily record collection of the invention based on peer-to-peer network is as shown in figure 3, find other side, and build between log system server
Vertical peer-to-peer network, wherein, reciprocity networking can use P2P Mechanism establishings.In peer-to-peer network, the log system of peer node takes each other
Be engaged between device will periodically intercommunication state, it is preferred that the state of log system server includes：Ability information, performance information
And range information, wherein：Ability information refers to the information of the type of log system server, ability and place network；Performance is believed
Breath refers to the load information of log system server, such as length of task queue etc.；Range information has reacted and reciprocity daily record
The distance between system server.
Log system server receives the RPC message that one or more business module servers are sent, and then, RPC is disappeared
Breath is sequentially placed into the task queue of itself according to the time of reception by the order in morning to evening.Business module is carried in RPC message
Log information caused by server.
When handling RPC message, log system server needs the state and reciprocity log system according to itself
The state of server performs, it is preferable that result includes two kinds of situations, first, by the log information recording in RPC message
To file, with as journal file；First, RPC message is directly transmitted to reciprocity log system server.
Log system server, can be according to following rule when the RPC message in itself task queue is handled：
According to the RPC message in the sequential processes task queue of priority from high to low；And/or in statistics task queue
The type of RPC message, RPC message corresponding to the most type of priority treatment message number；And/or priority treatment enters task
The earliest RPC message of queue.
Above-mentioned rule can be used in any combination, such as：, can be according to excellent for same type of multiple RPC message
The sequential processes of first level from high to low, can also priority treatment to enter task queue earliest；For the multiple of same priority
RPC message, can be earliest into task queue with priority treatment；, can be according to excellent for entering task queue time identical
The sequential processes of first level from high to low.
Whether log system server can handle RPC message according to the condition adjudgement of itself first, if it is, RPC is disappeared
Log information recording in breath is to file, and here, recorded file includes：It recorded local file or network file.If not,
Then RPC message is transmitted to reciprocity log system server by log system server.One log system server may have more
Individual reciprocity log system server, in selection, suitable equity can be selected according to the state of reciprocity log system server
Log system server, for example, that selection load is minimum or that chosen distance is nearest etc..Specifically, log system server
When whether first choice can handle RPC message according to the condition adjudgement of itself, and according to the choosing of the state of reciprocity log system server
When selecting suitable reciprocity log system server, it can be formulated according to the state of log system server corresponding one or more
Rule.
Reciprocity log system server receive forwarding come one or more RPC message after, also by its according to receive when
Between by morning to evening order be put into the task queue of itself, the operation of subsequent execution is as it was previously stated, here is omitted.
In this way, each log system server uploads to each self-recording journal file the file specified in reciprocity networking
Storage server.
In order to realize the above method, the daily record based on peer-to-peer network as shown in Figure 3 provided in an embodiment of the present invention is collected
In system, business module server and the respective function of log system server are as follows：
Business module server, corresponding log system service is sent to for producing log information, and by RPC message
Device；
Log system server, after the RPC message sent for business module server corresponding to receiving, according to itself
State and reciprocity log system server state, RPC message is handled, including：By the log information in RPC message
It recorded in file, with as journal file, or, RPC message is transmitted to reciprocity log system server；
Log system server, it is additionally operable to uploading to the journal file of self record into the document storage server specified.
The state of log system server includes：React the energy of the type of log system server, ability and place network
Distance between force information, the performance information for reacting log system server load and reaction and reciprocity log system server
Range information.
Log system server, it is additionally operable in log information recording to local file or network file.
Log system server or reciprocity log system server, itself is put into after the RPC message for being additionally operable to receive
In task queue.
Log system server, it is additionally operable to handle RPC message according to following rule：According to priority from high to low
Sequential processes task queue in RPC message；And/or in statistics task queue RPC message type, priority treatment message
RPC message corresponding to the most type of quantity；And/or enter task queue for same type of RPC message, priority treatment
Earliest RPC message.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.
Claims (10)
- A kind of 1. daily record gathering method based on peer-to-peer network, it is characterised in that establish by multiple log system server groups into Peer-to-peer network, this method also includes：Current log system server receives the remote procedure call protocol RPC message that corresponding business module server is sent Afterwards, according to the state of itself and the state of reciprocity log system server, the RPC message is handled, including：By described in Log information recording in RPC message is into file, with as journal file, or, the RPC message is transmitted to reciprocity day Will system server；Each self-recording journal file is uploaded to the file specified and deposited by each log system server in the peer-to-peer network Store up server.
- 2. the daily record gathering method based on peer-to-peer network according to claim 1, it is characterised in that log system server State includes：React the type, the ability information of ability and place network, reaction log system server of log system server The range information of distance between the performance information and reaction and reciprocity log system server of load.
- 3. the daily record gathering method based on peer-to-peer network according to claim 1, it is characterised in that described to remember log information Recording file includes：By in log information recording to local file or network file.
- 4. the daily record gathering method based on peer-to-peer network according to claims 1 to 3 is any, it is characterised in that described current After log system server or reciprocity log system server receive RPC message, RPC message is put into the task queue of itself In.
- 5. the daily record gathering method based on peer-to-peer network according to claim 4, it is characterised in that enter to the RPC message According to following rule during row processing：According to the RPC message in the sequential processes task queue of priority from high to low；And/or RPC disappears in statistics task queue The type of breath, RPC message corresponding to the most type of priority treatment message number；And/or for same type of RPC message, Priority treatment enters the earliest RPC message of task queue.
- A kind of 6. daily record gathering system based on peer-to-peer network, it is characterised in that including：Business module server, corresponding log system server is sent to for producing log information, and by RPC message；Log system server, after the RPC message sent for business module server corresponding to receiving, according to the shape of itself The state of state and reciprocity log system server, the RPC message is handled, including：By the daily record in the RPC message Information record is into file, with as journal file, or, the RPC message is transmitted to the reciprocity log system service Device；Log system server, it is additionally operable to uploading to the journal file of self record into the document storage server specified.
- 7. the daily record gathering system based on peer-to-peer network according to claim 6, it is characterised in that log system server State includes：React the type, the ability information of ability and place network, reaction log system server of log system server The range information of distance between the performance information and reaction and reciprocity log system server of load.
- 8. the daily record gathering system based on peer-to-peer network according to claim 6, it is characterised in that the log system service Device, it is additionally operable in log information recording to local file or network file.
- 9. the daily record gathering system based on peer-to-peer network according to claim 6 to 8 is any, it is characterised in that the daily record System server or the reciprocity log system server, the task queue of itself is put into after the RPC message for being additionally operable to receive In.
- 10. the daily record gathering system based on peer-to-peer network according to claim 9, it is characterised in that the log system clothes Business device, is additionally operable to handle RPC message according to following rule：According to the sequential processes task queue of priority from high to low In RPC message；And/or in statistics task queue RPC message type, the most type of priority treatment message number is corresponding RPC message；And/or for same type of RPC message, priority treatment enters the earliest RPC message of task queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210516908.9A CN103856534B (en) | 2012-12-05 | 2012-12-05 | A kind of daily record gathering method and system based on peer-to-peer network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210516908.9A CN103856534B (en) | 2012-12-05 | 2012-12-05 | A kind of daily record gathering method and system based on peer-to-peer network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103856534A CN103856534A (en) | 2014-06-11 |
| CN103856534B true CN103856534B (en) | 2018-02-13 |
Family
ID=50863741
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210516908.9A Active CN103856534B (en) | 2012-12-05 | 2012-12-05 | A kind of daily record gathering method and system based on peer-to-peer network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103856534B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106357454B (en) * | 2016-10-08 | 2020-06-16 | 武汉斗鱼网络科技有限公司 | Log system control device and control method thereof |
| CN108733698B (en) * | 2017-04-19 | 2023-08-08 | 腾讯科技（深圳）有限公司 | Log message processing method and background service system |
| CN108847989B (en) * | 2018-06-29 | 2021-07-06 | 杭州安恒信息技术股份有限公司 | Log processing method based on micro-service architecture, service system and electronic equipment |
| CN109710433B (en) * | 2018-12-30 | 2021-04-27 | 浙江蓝卓工业互联网信息技术有限公司 | Metadata-driven data processing method, device and system |
| CN109918282B (en) * | 2019-03-16 | 2023-02-24 | 河南工学院 | Computer log monitoring method and monitoring platform |
| CN110166298A (en) * | 2019-05-24 | 2019-08-23 | 和记奥普泰通信技术有限公司 | A kind of packetizing OTN, PTN, IPRAN and SPN control with management log realization system and method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101361054A (en) * | 2005-11-15 | 2009-02-04 | 蒂弗萨公司 | System for identifying the presence of peer-to-peer network software applications |
| CN101729581A (en) * | 2008-10-28 | 2010-06-09 | 华为技术有限公司 | Method, system and device for realizing re-orientation in P2P network |
| CN101800776A (en) * | 2010-02-26 | 2010-08-11 | 中国电信股份有限公司 | Network fusion and control method and system of CDN (Content Delivery Network) and P2P (Peer-to-Peer) |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120078765A1 (en) * | 2010-09-27 | 2012-03-29 | Ebay Inc. | Instant Financial Account Verification Using Direct Connect Data Communication Protocol And Open Financial Exchange Data-Stream Format |
-
2012
- 2012-12-05 CN CN201210516908.9A patent/CN103856534B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101361054A (en) * | 2005-11-15 | 2009-02-04 | 蒂弗萨公司 | System for identifying the presence of peer-to-peer network software applications |
| CN101729581A (en) * | 2008-10-28 | 2010-06-09 | 华为技术有限公司 | Method, system and device for realizing re-orientation in P2P network |
| CN101800776A (en) * | 2010-02-26 | 2010-08-11 | 中国电信股份有限公司 | Network fusion and control method and system of CDN (Content Delivery Network) and P2P (Peer-to-Peer) |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103856534A (en) | 2014-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103856534B (en) | A kind of daily record gathering method and system based on peer-to-peer network | |
| CN110048927B (en) | Communication method and communication device | |
| CN107818120A (en) | Data processing method and device based on big data | |
| CN103326896B (en) | The system and method for the information data that a kind of user of collection produces on the internet | |
| CN102244670B (en) | Idle node assistance method for P2P (peer-to-peer) file transmission | |
| CN103942664A (en) | Kitchen waste collection transportation supervision system based on vehicle networking technology | |
| CN103841018A (en) | Content centric network multiport forwarding method and router | |
| CN104866528B (en) | Multi-platform collecting method and system | |
| CN102065136A (en) | P2P (Peer-to-Peer) network safety data transmission method and system | |
| Germanus et al. | Increasing the resilience of critical scada systems using peer-to-peer overlays | |
| Yang et al. | SC-RPL: A social cognitive routing for communications in industrial Internet of Things | |
| JP5419135B2 (en) | Connection destination node selection method, node, and program | |
| US20110305234A1 (en) | Network positioning system and terminal positioning device | |
| JP4884520B2 (en) | Data collection method and system | |
| CN112333758B (en) | Stability monitoring management method, system and application of television gateway system | |
| US10254127B2 (en) | Method for remotely reading electric meters | |
| CN111538772B (en) | Data exchange processing method and device, electronic equipment and storage medium | |
| CN109831473A (en) | Logistics service providing method and equipment | |
| CN104199919A (en) | Method for achieving real-time reading of super-large-scale data | |
| JP2007274611A (en) | Conversion system selecting method | |
| CN103428231A (en) | Offline download method and system | |
| CN107749875B (en) | Big data artificial intelligence analytic system | |
| Zhai et al. | An improved DDS publish/subscribe automatic discovery algorithm | |
| KR102010570B1 (en) | Remote meter reading system with dcu cluster technique | |
| CN102263669B (en) | Network management system and method for controlled multicast system based on multicast conversion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |