CN103532755A - Method and system for processing distributed asynchronous events - Google Patents
Method and system for processing distributed asynchronous events Download PDFInfo
- Publication number
- CN103532755A CN103532755A CN201310476040.9A CN201310476040A CN103532755A CN 103532755 A CN103532755 A CN 103532755A CN 201310476040 A CN201310476040 A CN 201310476040A CN 103532755 A CN103532755 A CN 103532755A
- Authority
- CN
- China
- Prior art keywords
- log information
- event
- message
- processed
- central server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000012545 processing Methods 0.000 title claims abstract description 38
- 230000005540 biological transmission Effects 0.000 claims abstract description 15
- 238000003672 processing method Methods 0.000 claims description 19
- 238000012856 packing Methods 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 5
- 230000009545 invasion Effects 0.000 abstract description 6
- 230000007547 defect Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 241000243251 Hydra Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- QRXWMOHMRWLFEY-UHFFFAOYSA-N isoniazide Chemical compound NNC(=O)C1=CC=NC=C1 QRXWMOHMRWLFEY-UHFFFAOYSA-N 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001846 repelling effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Images
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention discloses a method and a system for processing distributed asynchronous events. The method comprises the following steps: processing to-be-processed events in each application server in a log mode, creating a first log message for each to-be-processed event and transmitting the first log message, wherein the first log message comprises a message type and message content at least; storing the first log message in a message queue in a central server; according to the message type of to-be-processed messages which are registered in the central server by each subscription server, respectively sending the messages corresponding to the registered message types in the message queue to the corresponding subscription servers; processing the messages by the subscription servers and returning message processing results to the central server. As log real-time transmission is combined with the message queue in the embodiment, whether the events are received by a receiving end and processed or not can be judged by the central server in real time in a low invasion way according to the message processing results, and defects in the prior art are better overcome.
Description
Technical field
The present invention relates to computer technology, be specifically related to a kind of processing method and system of distributed asynchronous event.
Background technology
Along with the development of network, more and more apply Network Based providing, even also Network Based providing of some critical business.These critical business such as online payments etc., have proposed higher requirement to the reliability of system.
Distributed processing system(DPS) is that many computers different location or that have difference in functionality or that have different pieces of information are coupled together with communication network, under the unified management of control system is controlled, completes in phase the computer system of information handling task.It is under the distributed communication environment communicating by network at the server of processing different affairs that distributed asynchronous event is processed, and transmitting terminal sends after the event of bearer messages, needn't wait for that receiving terminal is replied just can continue to carry out subsequent treatment; And receiving terminal is after receiving event, adopt asynchronous mode to process the process of event.
At present, can for example, by queue (Active MQ, Gearman) event transfer is carried out in service, reach the object of asynchronous process, wherein, Gearman is one and is used for work assignment to call and be more suitable for doing the machine of a certain work, do a certain work concomitantly to other machines, distributed earth, does load balancing or be used for calling the system of the function of other Languages between a plurality of calling; Also can pass through timing acquisition daily record analyzing and processing, reach the object of asynchronous process.
Distributed asynchronous event has dealt with transmitting terminal and must wait for that receiving terminal is handled and could continue the problem carried out after event, also solve the performance bottleneck problem of the event of a plurality of transmitting terminal requests of receiving terminal concurrent processing, therefore improved significantly load balancing and the efficiency of event handling.But, in distributed asynchronous event treatment system, the feedback of not waiting for receiving terminal due to transmitting terminal just continues execution, therefore, how to confirm that receiving terminal receives event and carried out processing the problem that must consider with regard to becoming distributed asynchronous event treatment system.
Summary of the invention
In view of the above problems, the present invention has been proposed to a kind of processing method and system of the distributed asynchronous event that overcomes the problems referred to above or address the above problem are at least in part provided.
According to one aspect of the present invention, a kind of processing method of distributed asynchronous event is provided, the method comprises:
In daily record mode, the event to be processed in each application server is processed, for each event to be processed creates the first log information, and transmitted the first log information, this first log information at least comprises News Category and message content;
This first log information is stored in the message queue in central server;
News Category according to from each Subscriber to the message to be processed of this central server registration, is sent to respectively corresponding Subscriber by the message corresponding with registered News Category in the message queue of this central server; And
This Subscriber is processed this message, and to this central server return messages result.
According to a further aspect in the invention, provide a kind for the treatment of system of distributed asynchronous event, this system comprises:
One or more application servers, each application server is suitable in daily record mode, event to be processed being processed, for each event to be processed creates the first log information, and transmit the first log information, this first log information at least comprises News Category and message content;
Central server, is suitable for receiving the first log information from each application server, and this first log information is stored in the message queue of this central server; And
One or more Subscribers, each Subscriber is suitable for registering to this central server the News Category of message to be processed, receive the message corresponding with registered News Category in that this central server transmits, this message queue, this message is processed, and to this central server return messages result.
The embodiment of the present invention, in daily record mode, the event to be processed in each application server is processed, for each event to be processed creates the first log information, and the first log information is stored in message queue, and the News Category of the message to be processed of registering to central server according to each Subscriber, the message corresponding with registered News Category in message queue is sent to respectively to corresponding Subscriber, Subscriber is to the backward central server return messages of above-mentioned Message Processing result, as can be seen here, the embodiment of the present invention combines daily record real-time Transmission with Message queue, make the central server can be according to this Message Processing result in real time, whether judge receiving terminal to low invasion receives event and processes, overcome preferably the defect of prior art.
Above-mentioned explanation is only the general introduction of technical solution of the present invention, in order to better understand technological means of the present invention, and can be implemented according to the content of specification, and for above and other objects of the present invention, feature and advantage can be become apparent, below especially exemplified by the specific embodiment of the present invention.
Accompanying drawing explanation
By reading below detailed description of the preferred embodiment, various other advantage and benefits will become cheer and bright for those of ordinary skills.Accompanying drawing is only for the object of preferred implementation is shown, and do not think limitation of the present invention.And in whole accompanying drawing, by identical reference symbol, represent identical parts.In the accompanying drawings:
Fig. 1 a shows the flow chart of the processing method of distributed asynchronous event according to an embodiment of the invention;
Fig. 1 b shows the flow chart of the thinning process of step S102 in Fig. 1 a of the present invention;
Fig. 2 shows the structural representation of the treatment system of distributed asynchronous event according to an embodiment of the invention;
Fig. 3 shows the structural representation of the treatment system of distributed asynchronous event in accordance with another embodiment of the present invention;
Fig. 4 shows the structural representation of message preprocessor according to an embodiment of the invention.
Embodiment
Exemplary embodiment of the present disclosure is described below with reference to accompanying drawings in more detail.Although shown exemplary embodiment of the present disclosure in accompanying drawing, yet should be appreciated that and can realize the disclosure and the embodiment that should do not set forth limits here with various forms.On the contrary, it is in order more thoroughly to understand the disclosure that these embodiment are provided, and can by the scope of the present disclosure complete convey to those skilled in the art.
By carry out event transfer with queue service, confirm that receiving terminal exists following problem to the reception of event and treatment state: application program is limited by the stability of queue self and the accessibility of network; For example, application system A directly accesses Gearman, if the reliability of Gearman itself is 90%, network-accessible is 90%, and the reliability of that original system A (being assumed to be 100%) drops to 90%*90%=81%.The decline of this reliability, produces serious influence to the reliability of business.
On the other hand, by timing acquisition daily record analysis, confirm that receiving terminal exists following problem to the reception of event and treatment state: although system be there is no to the impact of invasive and reliability, but because adopt the mode of automatic regular polling not reach real-time or punctual requirement, the longest meeting of time delay reaches more than 5 minutes.
Therefore,, how in real time low invasion ground confirms whether receiving terminal receives event and carried out processing and just become one of technical problem that distributed asynchronous event treatment system must solve.Fig. 1 a shows the flow chart of the processing method of distributed asynchronous event according to an embodiment of the invention, and as shown in Figure 1a, the method comprises:
Step S101, in daily record mode, the event to be processed in each application server is processed, for each event to be processed creates the first log information, and transmit the first log information, above-mentioned the first log information at least comprises News Category and message content;
Particularly, can for each event to be processed, create the first log information by calling syslog function, standard C function syslog under SDK (Hydar SDK) encapsulation unix, and the certain form of standard, concrete format description and journal format example are as follows:
// format description:
There is machine event classification (hydra_send|hydra_back) # event name # event id event data (JSON) in the time on date (iso reference format)
From above-mentioned format description, can find out that log information comprises one or more in following message: the time that event generates, generate host information, event category, event name, event identifier and the event content of event;
// journal format example
2012-07-24?12:09:24?w20.wg.zwt.qihoo.net?hydra_send#pay#21981248#{″money″:″10000″,″uid″:″138348497″,″time″:″1343102964″,″gkey″:″rxhzw″,″skey″:″S4″}
Wherein, the hydra_send in event classification refers to the daily record that sends to central server from application server;
In the present embodiment, open-source software syslog-ng that can be by transmitting daily record between each server is by the information router on application server to central server (Hydra SVR), and concrete example is as follows:
Step S102, the first log information is stored in the message queue in central server;
Syslog-ng on central server receives after the first log information from application server, and the first log information is stored in the message queue in central server, and as shown in Figure 1 b, this process comprises concrete thinning process:
Step S1021, from the first log information, extract event to be processed;
Step S1022, according to event configuration principle judgement, the first log information corresponding to the event with extracting packed or disappear and heavily process, if disappear, heavily process, turn to step S1023, if play processing, turn to step S1024;
Step S1023, the first log information corresponding to the event with extracting disappeared heavily and to be processed, and the first log information is stored in message queue;
This step S1023 specifically comprises: log information to be processed is stored in to the first buffer; Search in above-mentioned the first buffer and whether store the first log information, if there is no, above-mentioned the first log information is cached in above-mentioned the first buffer, and export in message queue, if existed, above-mentioned the first log information is not stored in above-mentioned the first buffer, does not transmit it in above-mentioned message queue yet; The first log information a and the first log information b for example in current the first buffer, have been stored, if the first log information corresponding to the current event with extracting is the first log information c, owing to there not being the first log information c in current first memory, this first log information c is cached in the first buffer, and exports in message queue; If the first log information corresponding to the current event with extracting is the first log information b, owing to there being the first log information b in current first memory, above-mentioned the first log information b is not stored in above-mentioned the first buffer, does not transmit it in above-mentioned message queue yet;
In addition, the method can also comprise: clocked flip and remove the data in the first buffer, thus can avoid data cached excessive situation to occur;
Step S1024, to the processing of packing of the first log information corresponding to the event with extracting, and the first log information is stored in message queue.
This step S1024 specifically comprises: buffer memory log information to be processed, after the log information quantity to be processed of institute's buffer memory reaches default maximum, by the synthetic log information of the log information of institute's buffer memory, and outputs in message queue.
In addition, under the log information quantity of buffer memory does not reach default peaked situation, also can clocked flip and by the synthetic log information of the log information of buffer memory, and output in above-mentioned message queue, within the scheduled time, timer can clocked flip, and synthetic one of the log information that does not reach default maximum number is exported in message queue.
It should be noted that, above-mentioned steps S1023 and step S1024 there is no strict execution sequence, for example, both can only perform step S1023, also can only perform step S1024, perform step S1024 after can also first performing step S1023.
Step S103, the News Category according to each Subscriber to the message to be processed of above-mentioned central server registration, be sent to respectively corresponding Subscriber by the message corresponding with registered News Category in the message queue of above-mentioned central server;
Center processor monitoring information, is sent to respectively corresponding Subscriber by message corresponding with registered News Category in message queue, is specifically sent in the SDK of corresponding Subscriber.
Step S104, Subscriber are processed above-mentioned message, and to above-mentioned central server return messages result.
Subscriber processes to generate the second log information in daily record mode to message result after message is disposed, and the second daily record transmission of messages is arrived to central server; Specifically by the SDK on Subscriber, Message Processing result is fed back to central server in daily record mode, this daily record can adopt the form defining in step S101, format description is not repeated herein, and concrete journal format example is as follows:
// journal format example
2012-07-2416:52:41w1.wg.ccp.qihoo.net
hydra_back#gamezone_history.event_farm#28886141hero#
{″qid″:″28886141″,″gkey″:″hero″,″skey″:null,″create_time″:1343119961}
Wherein, Hydra_back refers to the daily record that feeds back to central server after Subscriber is processed.
The processing method of above-mentioned distributed asynchronous event, in daily record mode, the event to be processed in each application server is processed, for each event to be processed creates the first log information, and the first log information is stored in message queue, and the News Category of the message to be processed of registering to central server according to each Subscriber, the message corresponding with registered News Category in message queue is sent to respectively to corresponding Subscriber, Subscriber is to the backward central server return messages of above-mentioned Message Processing result, central server can be in real time according to this Message Processing result, whether judge receiving terminal to low invasion receives event and processes, overcome preferably the defect of prior art.
Fig. 2 shows the treatment system of distributed asynchronous event according to an embodiment of the invention, as shown in Figure 2, the treatment system of this distributed asynchronous event comprises: one or more application servers 210, central server 220 and one or more Subscriber 230, wherein:
Each application server is suitable in daily record mode, event to be processed being processed, and for each event to be processed creates the first log information, and transmits the first log information, and above-mentioned the first log information at least comprises News Category and message content;
Central server is suitable for receiving the first log information from each application server, and above-mentioned the first log information is stored in the message queue of above-mentioned central server;
Each Subscriber is suitable for registering to above-mentioned central server the News Category of message to be processed, receive the message corresponding with registered News Category in message queue that above-mentioned central server transmits, above-mentioned, above-mentioned message is processed, and to above-mentioned central server return messages result.
Each application server in above-mentioned one or more application server is trigger event safely, and do not worry relying on the stability that third party serves reduction system, as shown in Figure 3, each application server can comprise: event generating 2110, is suitable for generating first log information corresponding with event to be processed in daily record mode; The first log transmission unit 2120, is suitable for when above-mentioned event generating generates the first log information, and the first generated log information is transferred to above-mentioned central server.Meanwhile, above-mentioned subscription service implement body processes to generate the second log information in daily record mode to message result, and above-mentioned the second daily record transmission of messages is arrived to above-mentioned central server.Wherein, above-mentioned the first log information and above-mentioned the second log information comprise one or more in following message: host information, event category, event name, event identifier and the event content of the time that event generates, generation event; Concrete format description can, referring to the associated description in step S101 in embodiment of the method, not repeat herein.
As shown in Figure 3, each Subscriber in above-mentioned one or more Subscriber can comprise Registration Module 2300, Event receiver 2310, event handler 2320 and the second daily record transmission unit 2330; Wherein: Registration Module is suitable for registering to above-mentioned central server the News Category of message to be processed; Event receiver is suitable for receiving the message corresponding with registered News Category in message queue that above-mentioned central server transmits, above-mentioned, extracts event to be processed from above-mentioned message; The event that event handler is suitable for above-mentioned Event receiver to extract is processed, and obtains Message Processing result; The second daily record transmission unit is suitable for, in the mode of the second log information, the above-mentioned Message Processing result from above-mentioned event handler is transferred to above-mentioned central server.In addition, above-mentioned central server can comprise Journal receiver 2210, message distributor 2230 and queue memory 2240, wherein: Journal receiver is suitable for receiving from the first log information of above-mentioned application server and from the second log information of above-mentioned Subscriber; Queue memory is suitable for above-mentioned the first log information to store in above-mentioned message queue; Message distributor is suitable for receiving the News Category from the message to be processed of above-mentioned Subscriber, monitors above-mentioned message queue, and distributes the extremely corresponding Subscriber of corresponding message in above-mentioned message queue according to the News Category of message to be processed.Further, above-mentioned central server can also comprise: message preprocessor 2220; This message preprocessor is couple between above-mentioned Journal receiver and above-mentioned queue memory, be suitable for extracting event to be processed from the first log information from above-mentioned Journal receiver, according to event configuration principle pair first log information corresponding with the event extracting, pack or disappear and heavily process, and treated message is sent to above-mentioned queue memory.
Particularly, the structure of above-mentioned message preprocessor as shown in Figure 4, comprise the first buffer 410, disappear heavy processing unit 420 and packing processing unit 440, wherein: the first buffer is suitable for buffer memory from the log information to be processed of above-mentioned Journal receiver; The heavy processing unit that disappears is suitable for searching in the first buffer whether store the first log information, if there is no, above-mentioned the first log information is cached in above-mentioned the first buffer, and above-mentioned the first log information is transferred to above-mentioned queue memory; If existed, above-mentioned the first log information is not stored in above-mentioned the first buffer, do not transmit it to above-mentioned queue memory yet; Packing processing unit is suitable for buffer memory log information to be processed, after the log information quantity to be processed of institute's buffer memory reaches default maximum, by the synthetic log information of the log information of institute's buffer memory, and exports above-mentioned queue memory to.In addition, this message preprocessor can also comprise: timer 430, and this timer is suitable for removing the data in above-mentioned the first buffer for clocked flip, thereby can avoid the excessive situation of data to occur; Also can be in above-mentioned packing processing unit the log information quantity of buffer memory do not reach in default peaked situation, clocked flip is the synthetic log information of the log information of buffer memory in above-mentioned packing processing unit, and exports to above-mentioned queue memory.
The event handler procedure of the treatment system of the distributed asynchronous event of structure, as shown in Fig. 1 a-1b, does not repeat herein as in Figure 2-4.
The treatment system of above-mentioned distributed asynchronous event, by application server, be that each event to be processed creates the first log information, and the first log information is transferred to central server, central server is stored in message queue after receiving the first log information, and the News Category of the message to be processed of registering to central server according to each Subscriber, the message corresponding with registered News Category in message queue is sent to respectively to corresponding Subscriber, Subscriber is to the backward central server return messages of above-mentioned Message Processing result, central server can be in real time by this Message Processing result, whether judge receiving terminal to low invasion receives event and processes.
As can be seen here, above-mentioned treatment system combines daily record real-time Transmission with Message queue, can carry out in real time event transfer (second level), application system is not had invasive and do not reduce the stability of application system, and make central server in real time, low invasion to judge receiving terminal according to this Message Processing result and whether receive event and process, overcome preferably the defect of prior art.
The algorithm providing at this is intrinsic not relevant to any certain computer, virtual system or miscellaneous equipment with demonstration.Various general-purpose systems also can with based on using together with this teaching.According to description above, it is apparent constructing the desired structure of this type systematic.In addition, the present invention is not also for any certain programmed language.It should be understood that and can utilize various programming languages to realize content of the present invention described here, and the description of above language-specific being done is in order to disclose preferred forms of the present invention.
In the specification that provided herein, a large amount of details have been described.Yet, can understand, embodiments of the invention can not put into practice in the situation that there is no these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand one or more in each inventive aspect, in the above in the description of exemplary embodiment of the present invention, each feature of the present invention is grouped together into single embodiment, figure or sometimes in its description.Yet, the method for the disclosure should be construed to the following intention of reflection: the present invention for required protection requires than the more feature of feature of clearly recording in each claim.Or rather, as reflected in claims below, inventive aspect is to be less than all features of disclosed single embodiment above.Therefore, claims of following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and can the module in the equipment in embodiment are adaptively changed and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and can put them into a plurality of submodules or subelement or sub-component in addition.At least some in such feature and/or process or unit are mutually repelling, and can adopt any combination to combine all processes or the unit of disclosed all features in this specification (comprising claim, summary and the accompanying drawing followed) and disclosed any method like this or equipment.Unless clearly statement in addition, in this specification (comprising claim, summary and the accompanying drawing followed) disclosed each feature can be by providing identical, be equal to or the alternative features of similar object replaces.
In addition, those skilled in the art can understand, although embodiment more described herein comprise some feature rather than further feature included in other embodiment, the combination of the feature of different embodiment means within scope of the present invention and forms different embodiment.For example, in the following claims, the one of any of embodiment required for protection can be used with compound mode arbitrarily.
All parts embodiment of the present invention can realize with hardware, or realizes with the software module moved on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that and can use in practice microprocessor or digital signal processor (DSP) to realize the some or all functions according to the some or all parts in the treatment system of the distributed asynchronous event of the embodiment of the present invention.The present invention for example can also be embodied as, for carrying out part or all equipment or device program (, computer program and computer program) of method as described herein.Realizing program of the present invention and can be stored on computer-readable medium like this, or can there is the form of one or more signal.Such signal can be downloaded and obtain from internet website, or provides on carrier signal, or provides with any other form.
It should be noted above-described embodiment the present invention will be described rather than limit the invention, and those skilled in the art can design alternative embodiment in the situation that do not depart from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and is not listed as element or step in the claims.Being positioned at word " " before element or " one " does not get rid of and has a plurality of such elements.The present invention can be by means of including the hardware of some different elements and realizing by means of the computer of suitably programming.In having enumerated the unit claim of some devices, several in these devices can be to carry out imbody by same hardware branch.The use of word first, second and C grade does not represent any order.Can be title by these word explanations.
The processing method that the invention also discloses A1, a kind of distributed asynchronous event, comprising:
In daily record mode, the event to be processed in each application server is processed, for each event to be processed creates the first log information, and transmitted the first log information, described the first log information at least comprises News Category and message content;
Described the first log information is stored in the message queue in central server;
News Category according to from each Subscriber to the message to be processed of described central server registration, is sent to respectively corresponding Subscriber by the message corresponding with registered News Category in the message queue of described central server; And
Described Subscriber is processed described message, and to described central server return messages result.
A2. according to the processing method described in A1, wherein, described Subscriber is processed described message, and to described central server return messages result, comprising:
Described Subscriber processes to generate the second log information in daily record mode to message result, and described the second daily record transmission of messages is arrived to described central server.
A3. according to the processing method described in A1 or A2, wherein, described the first log information and described the second log information comprise one or more in following: the time that event generates, generate host information, event category, event name, event identifier and the event content of event.
A4. according to the processing method described in A1, wherein, described the first log information is stored in the message queue in central server, comprising:
From described the first log information, extract event to be processed;
According to event configuration principle pair first log information corresponding with the event extracting, pack and/or disappear and heavily process, then store in message queue.
A5. according to the processing method described in A4, wherein, described the first log information corresponding to the event with extracting disappeared and heavily processed, comprising:
Log information to be processed is stored in to the first buffer;
Search in described the first buffer and whether store the first log information, if there is no, described the first log information is cached in described the first buffer, and export in described message queue, if existed, described the first log information is not stored in described the first buffer, does not transmit it in described message queue yet;
Wherein, described to the processing of packing of the first log information corresponding to the event with extracting, comprise: buffer memory log information to be processed, after the log information quantity to be processed of institute's buffer memory reaches default maximum, by the synthetic log information of the log information of institute's buffer memory, and output in described message queue.
A6. according to the processing method described in A5, the method also comprises:
Clocked flip and remove the data in described the first buffer; And/or
Log information quantity at buffer memory does not reach in described default peaked situation, clocked flip and by the synthetic log information of the log information of buffer memory, and output in described message queue.
The treatment system that the invention also discloses a B7. distributed asynchronous event, comprising:
One or more application servers, each application server is suitable in daily record mode, event to be processed being processed, for each event to be processed creates the first log information, and transmit the first log information, described the first log information at least comprises News Category and message content;
Central server, is suitable for receiving the first log information from each application server, and described the first log information is stored in the message queue of described central server; And
One or more Subscribers, each Subscriber is suitable for registering to described central server the News Category of message to be processed, receive the message corresponding with registered News Category in message queue that described central server transmits, described, described message is processed, and to described central server return messages result.
B8. according to the treatment system described in B7, wherein, each application server in described one or more application server comprises:
Event generating, is suitable for generating first log information corresponding with event to be processed in daily record mode;
The first log transmission unit, is suitable for when described event generating generates the first log information, and the first generated log information is transferred to described central server.
B9. according to the treatment system described in B7, wherein, described Subscriber, is specifically suitable for: in daily record mode, message result is processed to generate the second log information, and described the second daily record transmission of messages is arrived to described central server.
B10. according to the treatment system described in B9, wherein, described the first log information and described the second log information comprise one or more in following: the time that event generates, generate host information, event category, event name, event identifier and the event content of event.
B11. according to the treatment system described in B7, wherein, each Subscriber in described one or more Subscriber comprises:
Registration Module, is suitable for registering to described central server the News Category of message to be processed;
Event receiver, is suitable for receiving the message corresponding with registered News Category in message queue that described central server transmits, described, extracts event to be processed from described message;
Event handler, the event that is suitable for described Event receiver to extract is processed, and obtains Message Processing result;
The second daily record transmission unit, is suitable for, in the mode of the second log information, the described Message Processing result from described event handler is transferred to described central server.
B12. the treatment system as described in any in B7-B11, wherein, described central server comprises:
Journal receiver, is suitable for receiving from the first log information of described application server and from the second log information of described Subscriber;
Queue memory, is suitable for described the first log information to store in described message queue;
Message distributor, is suitable for receiving the News Category from the message to be processed of described Subscriber, monitors described message queue, and distributes the extremely corresponding Subscriber of corresponding message in described message queue according to the News Category of message to be processed.
B13. according to the treatment system described in B12, wherein, described central server also comprises:
Message preprocessor, be couple between described Journal receiver and described queue memory, be suitable for extracting event to be processed from the first log information from described Journal receiver, according to event configuration principle pair first log information corresponding with the event extracting, pack or disappear and heavily process, and treated message is sent to described queue memory.
B14. according to the treatment system described in B13, wherein, described message preprocessor comprises:
The first buffer, is suitable for buffer memory from the log information to be processed of described Journal receiver;
Disappear and weigh processing unit, be suitable for searching in the first buffer whether store the first log information, if there is no, described the first log information is cached in described the first buffer, and described the first log information is transferred to described queue memory; If existed, described the first log information is not stored in described the first buffer, do not transmit it to described queue memory yet; And/or
Packing processing unit, is suitable for buffer memory log information to be processed, after the log information quantity to be processed of institute's buffer memory reaches default maximum, by the synthetic log information of the log information of institute's buffer memory, and exports described queue memory to.
B15. according to the system described in B14, wherein, described message preprocessor also comprises:
Timer, is suitable for removing the data in described the first buffer for clocked flip; And/or the log information quantity of buffer memory does not reach in default peaked situation in described packing processing unit, clocked flip is the synthetic log information of the log information of buffer memory in described packing processing unit, and exports to described queue memory.
Claims (10)
1. a processing method for distributed asynchronous event, comprising:
In daily record mode, the event to be processed in each application server is processed, for each event to be processed creates the first log information, and transmitted the first log information, described the first log information at least comprises News Category and message content;
Described the first log information is stored in the message queue in central server;
News Category according to from each Subscriber to the message to be processed of described central server registration, is sent to respectively corresponding Subscriber by the message corresponding with registered News Category in the message queue of described central server; And
Described Subscriber is processed described message, and to described central server return messages result.
2. processing method according to claim 1, wherein, described Subscriber is processed described message, and to described central server return messages result, comprising:
Described Subscriber processes to generate the second log information in daily record mode to message result, and described the second daily record transmission of messages is arrived to described central server.
3. processing method according to claim 1 and 2, wherein, described the first log information and described the second log information comprise one or more in following: the time that event generates, generate host information, event category, event name, event identifier and the event content of event.
4. processing method according to claim 1, wherein, stores described the first log information in the message queue in central server into, comprising:
From described the first log information, extract event to be processed;
According to event configuration principle pair first log information corresponding with the event extracting, pack and/or disappear and heavily process, then store in message queue.
5. processing method according to claim 4, wherein, describedly disappears and heavily processes the first log information corresponding to the event with extracting, and comprising:
Log information to be processed is stored in to the first buffer;
Search in described the first buffer and whether store the first log information, if there is no, described the first log information is cached in described the first buffer, and export in described message queue, if existed, described the first log information is not stored in described the first buffer, does not transmit it in described message queue yet;
Wherein, described to the processing of packing of the first log information corresponding to the event with extracting, comprise: buffer memory log information to be processed, after the log information quantity to be processed of institute's buffer memory reaches default maximum, by the synthetic log information of the log information of institute's buffer memory, and output in described message queue.
6. processing method according to claim 5, the method also comprises:
Clocked flip and remove the data in described the first buffer; And/or
Log information quantity at buffer memory does not reach in described default peaked situation, clocked flip and by the synthetic log information of the log information of buffer memory, and output in described message queue.
7. a treatment system for distributed asynchronous event, comprising:
One or more application servers, each application server is suitable in daily record mode, event to be processed being processed, for each event to be processed creates the first log information, and transmit the first log information, described the first log information at least comprises News Category and message content;
Central server, is suitable for receiving the first log information from each application server, and described the first log information is stored in the message queue of described central server; And
One or more Subscribers, each Subscriber is suitable for registering to described central server the News Category of message to be processed, receive the message corresponding with registered News Category in message queue that described central server transmits, described, described message is processed, and to described central server return messages result.
8. treatment system according to claim 7, wherein, each application server in described one or more application server comprises:
Event generating, is suitable for generating first log information corresponding with event to be processed in daily record mode;
The first log transmission unit, is suitable for when described event generating generates the first log information, and the first generated log information is transferred to described central server.
9. treatment system according to claim 7, wherein, described Subscriber, is specifically suitable for: in daily record mode, message result is processed to generate the second log information, and by described the second daily record transmission of messages to described central server.
10. treatment system according to claim 9, wherein, described the first log information and described the second log information comprise one or more in following: the time that event generates, generate host information, event category, event name, event identifier and the event content of event.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310476040.9A CN103532755B (en) | 2013-10-12 | 2013-10-12 | The processing method of distributed asynchronous event and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310476040.9A CN103532755B (en) | 2013-10-12 | 2013-10-12 | The processing method of distributed asynchronous event and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103532755A true CN103532755A (en) | 2014-01-22 |
| CN103532755B CN103532755B (en) | 2016-07-06 |
Family
ID=49934456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310476040.9A Active CN103532755B (en) | 2013-10-12 | 2013-10-12 | The processing method of distributed asynchronous event and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103532755B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103856353A (en) * | 2014-03-06 | 2014-06-11 | 上海爱数软件有限公司 | Service log data access and statistic analysis method and device |
| CN105159657A (en) * | 2015-06-12 | 2015-12-16 | 北京京东尚科信息技术有限公司 | Method and system for processing message |
| CN105227367A (en) * | 2015-10-15 | 2016-01-06 | 北京英诺威尔科技股份有限公司 | A kind of warning analysis processing method of low delay |
| CN106776079A (en) * | 2016-12-28 | 2017-05-31 | Tcl集团股份有限公司 | A kind of method and system for managing user data |
| CN107369012A (en) * | 2016-07-05 | 2017-11-21 | 康存乐付保数据科技（上海）有限公司 | A kind of method for processing payment information and system and the prepaid card processing method with performance bond insurance mechanism |
| CN110113232A (en) * | 2019-03-27 | 2019-08-09 | 深圳绿米联创科技有限公司 | Apparatus control method, device, system, electronic equipment and storage medium |
| CN110288486A (en) * | 2019-05-22 | 2019-09-27 | 深圳壹账通智能科技有限公司 | Asynchronous compensation method, device, equipment and the storage medium of real-time processing data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102611642A (en) * | 2012-02-27 | 2012-07-25 | 杭州闪亮科技有限公司 | System for processing nonsynchronous message and method for system to send message and monitor processing task |
| CN102664947A (en) * | 2012-04-18 | 2012-09-12 | 迈普通信技术股份有限公司 | Notification distribution method for distributed system |
| CN102685173A (en) * | 2011-04-14 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Asynchronous task distribution system and scheduling distribution computing unit |
-
2013
- 2013-10-12 CN CN201310476040.9A patent/CN103532755B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102685173A (en) * | 2011-04-14 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Asynchronous task distribution system and scheduling distribution computing unit |
| CN102611642A (en) * | 2012-02-27 | 2012-07-25 | 杭州闪亮科技有限公司 | System for processing nonsynchronous message and method for system to send message and monitor processing task |
| CN102664947A (en) * | 2012-04-18 | 2012-09-12 | 迈普通信技术股份有限公司 | Notification distribution method for distributed system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103856353A (en) * | 2014-03-06 | 2014-06-11 | 上海爱数软件有限公司 | Service log data access and statistic analysis method and device |
| CN105159657A (en) * | 2015-06-12 | 2015-12-16 | 北京京东尚科信息技术有限公司 | Method and system for processing message |
| CN105227367A (en) * | 2015-10-15 | 2016-01-06 | 北京英诺威尔科技股份有限公司 | A kind of warning analysis processing method of low delay |
| CN107369012A (en) * | 2016-07-05 | 2017-11-21 | 康存乐付保数据科技（上海）有限公司 | A kind of method for processing payment information and system and the prepaid card processing method with performance bond insurance mechanism |
| CN106776079A (en) * | 2016-12-28 | 2017-05-31 | Tcl集团股份有限公司 | A kind of method and system for managing user data |
| CN110113232A (en) * | 2019-03-27 | 2019-08-09 | 深圳绿米联创科技有限公司 | Apparatus control method, device, system, electronic equipment and storage medium |
| CN110288486A (en) * | 2019-05-22 | 2019-09-27 | 深圳壹账通智能科技有限公司 | Asynchronous compensation method, device, equipment and the storage medium of real-time processing data |
| WO2020233310A1 (en) * | 2019-05-22 | 2020-11-26 | 深圳壹账通智能科技有限公司 | Asynchronous compensation method and apparatus for real-time processing data, device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103532755B (en) | 2016-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103530255A (en) | Distributed asynchronous event processing method and system | |
| CN103532755A (en) | Method and system for processing distributed asynchronous events | |
| CN104620558B (en) | System and method for supporting the message in distributed data grid cluster to pre-process | |
| CN103207785B (en) | The processing method of data download request, Apparatus and system | |
| CN112632566B (en) | Vulnerability scanning method and device, storage medium and electronic equipment | |
| CN111818158B (en) | Gateway control method, device, electronic equipment and storage medium | |
| CN103577251A (en) | Event based Internet computing processing system and method | |
| CN105373420A (en) | Data transmission method and apparatus | |
| CN113630310B (en) | Distributed high-availability gateway system | |
| CN105095299A (en) | Picture capturing method and system | |
| CN103559239B (en) | The processing method and system and task server of picture | |
| CN114338801A (en) | Lightweight publishing and subscribing method and system suitable for intelligent measuring equipment, equipment and storage medium | |
| CN110247926A (en) | A kind of exchange method and system | |
| CN107105004B (en) | Cross-platform service calling method and device | |
| CN113965628A (en) | Message scheduling method, server and storage medium | |
| CN102761545A (en) | Service processing method, service processor and service processing system | |
| CN104394095A (en) | Data transmission method, data transmission system and source server | |
| CN114050909B (en) | Exercise method, system and electronic equipment for simulating mail | |
| CN115167985A (en) | Virtualized computing power providing method and system | |
| CN114039963A (en) | Data transmission method, device, electronic device and storage medium | |
| CN109460215B (en) | Application control method and device | |
| CN110535743B (en) | Data packet processing method and device, storage medium and electronic device | |
| CN108055305B (en) | Storage expansion method and storage expansion device | |
| CN113760693A (en) | Method and apparatus for local debugging of microservice systems | |
| CN103297455B (en) | Connection sharing method between browser and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20160511 Address after: 100041 room, room 2, building 2, No. 17 West Road, Beijing, Shijingshan District, China Applicant after: Beijing Shijie Xinghui Science and Technology Co., Ltd. Address before: 100088 Beijing city Xicheng District xinjiekouwai Street 28, block D room 112 (Desheng Park) Applicant before: Beijing Qihu Technology Co., Ltd. Applicant before: Qizhi Software (Beijing) Co., Ltd. |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |