CN105227662A - Message treatment method, device and system - Google Patents
Message treatment method, device and system Download PDFInfo
- Publication number
- CN105227662A CN105227662A CN201510646199.XA CN201510646199A CN105227662A CN 105227662 A CN105227662 A CN 105227662A CN 201510646199 A CN201510646199 A CN 201510646199A CN 105227662 A CN105227662 A CN 105227662A
- Authority
- CN
- China
- Prior art keywords
- message
- consumer
- module
- cluster
- record sheet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kind of message treatment method, device and system, described method comprises: message consumer obtains the message that server cluster pushes; According to preset record sheet, judge whether to receive described message, in described record sheet, record the message that described message consumer had received; If do not receive described message, then processed described message.The invention also discloses a kind of message processing apparatus and message handling system.Present invention achieves the filtration to repetition message, only process non-repetitive message, decrease resource occupation, Message Processing efficiency is high, improves the validity of service.
Description
Technical field
The present invention relates to technical field of information processing, particularly relate to a kind of message treatment method, device and system.
Background technology
In current message handling system, the message producer sends message, is received and be pushed to message consumer to process by message server cluster.
When Message Processing amount is large or when reaching Message handling request peak, the message producer sends a large amount of message, after message server cluster receives a large amount of message, message consumer is sent to after rough distribution, the stability of Message Processing can not be ensured, the message order causing message consumer to receive is mixed and disorderly, and treatment effeciency is low.
And, when Message Processing amount is large or when reaching Message handling request peak, the frequency duplicating message is higher, after now message consumer receives message, the message that reprocessing is identical, causes the waste of resource transfer, disposal ability declines, efficiency is low, cannot complete the process of message in time.
Summary of the invention
Main purpose of the present invention is to provide a kind of message treatment method, device and system, and the process being intended to solve repetition message causes the inefficient technical problem of Message Processing.
For achieving the above object, the invention provides a kind of message treatment method, described message treatment method comprises the following steps:
Message consumer obtains the message that server cluster pushes;
According to preset record sheet, judge whether to receive described message, in described record sheet, record the message that described message consumer had received;
If do not receive described message, then processed described message.
In one embodiment, described according to preset record sheet, judge whether to receive described message, after recording the step of the message that described message consumer had received in described record sheet, also comprise:
If received described message, then deleted described message, and proceeded to execution step: message consumer obtained the message that server cluster pushes.
In one embodiment, described according to preset record sheet, judge whether to receive described message, after recording the step of the message that described message consumer had received in described record sheet, also comprise:
If do not receive described message, then in described record sheet, recorded described message, to upgrade described record sheet.
In one embodiment, do not receive described message if described, then, after processing the step of described message, also comprised:
According to the duplicate removal effect preset and/or processing speed, delete the record in described record sheet in preset time range at the timing node preset.
In one embodiment, described message consumer also comprises after obtaining the step of the message that server cluster pushes:
To described server set pocket transmission response, and obtain the next message of described server cluster propelling movement.
In addition, for achieving the above object, the present invention also provides a kind of message processing apparatus, and described message processing apparatus comprises:
Acquisition module, obtains the message of server cluster propelling movement for message consumer;
Judge module, for according to preset record sheet, judges whether to receive described message, records the message that described message consumer had received in described record sheet;
Processing module, if for not receiving described message, then process described message.
In one embodiment, described message processing apparatus also comprises:
Duplicate removal module, if for receiving described message, then delete described message;
Described acquisition module, also obtains the next message of server cluster propelling movement for message consumer.
In one embodiment, described message processing apparatus also comprises:
Update module, if for not receiving described message, then recorded described non-duplicate message, to upgrade described record sheet in described record sheet.
In one embodiment, described message processing apparatus also comprises:
Removing module, for according to the duplicate removal effect preset and/or processing speed, deletes the record in described record sheet in preset time range at the timing node preset.
In one embodiment, described message processing apparatus also comprises:
Responder module, for replying to described server set pocket transmission.
In addition, for achieving the above object, the present invention also provides a kind of message handling system, and described message handling system comprises message producer cluster, server cluster and message consumer's cluster, wherein:
Described message producer cluster, gives described server cluster for the production of information and sending;
Described server cluster, for receiving the message that described message producer cluster sends, and is pushed to described message consumer's cluster;
Described message consumer's cluster, comprises acquisition module, judge module, processing module, duplicate removal module, update module, removing module and responder module.
In one embodiment, described server cluster comprises one or more server, wherein:
Described server comprises switch module and Queue module;
Described switch module, for receiving the message that described message producer cluster sends, sends to the Queue module with the described match messages received by the described message received;
Described Queue module, for receiving the message that described switch module sends, being pushed to described message consumer's cluster by the described message received successively by the rule preset, after receiving the response of described message consumer's cluster, deleting the corresponding message of described response.
A kind of message treatment method that the embodiment of the present invention proposes, device and system, by obtaining the message that server cluster pushes; According to preset record sheet, judge whether to receive described message, in described record sheet, record the message that described message consumer had received; If do not receive described message, then processed described message, and avoided that reprocessing is carried out to the message received and take resource.Present invention achieves the filtration to repetition message, only process non-repetitive message, decrease resource occupation, Message Processing efficiency is high, improves the validity of service.
Accompanying drawing explanation
Fig. 1 is the schematic flow sheet of message treatment method first embodiment of the present invention;
Fig. 2 is the schematic flow sheet of message treatment method second embodiment of the present invention;
Fig. 3 is the schematic flow sheet of message treatment method of the present invention 3rd embodiment;
Fig. 4 is the schematic flow sheet of message treatment method of the present invention 4th embodiment;
Fig. 5 is the schematic flow sheet of message treatment method of the present invention 5th embodiment;
Fig. 6 is the high-level schematic functional block diagram of message processing apparatus first embodiment of the present invention;
Fig. 7 is the high-level schematic functional block diagram of message processing apparatus second embodiment of the present invention;
Fig. 8 is the high-level schematic functional block diagram of message processing apparatus of the present invention 3rd embodiment;
Fig. 9 is the high-level schematic functional block diagram of message processing apparatus of the present invention 4th embodiment;
Figure 10 is the high-level schematic functional block diagram of message processing apparatus of the present invention 5th embodiment;
Figure 11 is the module diagram of message handling system first embodiment of the present invention;
Figure 12 is the schematic diagram of a kind of embodiment of message handling system of the present invention;
Figure 13 is the module diagram of message handling system second embodiment of the present invention;
Figure 14 is the schematic diagram of a kind of application in the embodiment of the present invention.
The realization of the object of the invention, functional characteristics and advantage will in conjunction with the embodiments, are described further with reference to accompanying drawing.
Embodiment
Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.
The primary solutions of the embodiment of the present invention is: obtain the message that server cluster pushes; According to preset record sheet, judge whether to receive described message, in described record sheet, record the message that described message consumer had received; If do not receive described message, then processed described message.
Due to the prior art message producer send a large amount of message time, message consumer cannot get rid of the message of repetition, and all process all message, cause the wasting of resources, treatment effeciency is low.
The invention provides a solution, repetition message message consumer being removed receive, only processes once repetition message, promotes treatment effeciency.
With reference to Fig. 1, message treatment method first embodiment of the present invention provides a kind of message treatment method, and described message treatment method comprises:
Step S10, message consumer obtain the message that server cluster pushes.
The present invention is mainly used in the Message Processing pattern of the message producer, server cluster and message consumer.In the present embodiment, get one or more message producer cluster as the message producer, each message producer cluster can comprise one or more producer, the producer for the production of and send message; Get one or more message consumer's cluster as message consumer, each message consumer's cluster can comprise one or more consumer, and consumer is for the treatment of message.
In the message that the message producer generates, after being pushed to message consumer by server cluster, message consumer obtains the message that server cluster pushes and to go forward side by side row relax.
Concrete, as a kind of execution mode, message consumer is the message that pushes of reception server cluster successively, obtains the message that server cluster pushes.
Using the order of message consumer's receipt message as follow-up order of carrying out Message Processing.
Step S20, according to preset record sheet, judge whether to receive described message, in described record sheet, record the message that described message consumer had received.
After acquisition message, message consumer judges the current message obtained.
Concrete, as a kind of execution mode, first, message consumer obtains the keyword of current message.Wherein, the sign a piece of news that the keyword of message can be unique, the message that keyword is identical is repetition message, and the content of repetition message is identical.
Then, message consumer searches preset record sheet, searches the Message Record whether having keyword identical.Wherein, preset record sheet, with the form of Message Record, records the paid-in message of message consumer.Message consumer, by being mated with the keyword of current message by the keyword in Message Record, searches and whether has the Message Record identical with current message keyword.
If find the Message Record that keyword is identical in preset record sheet, then judge to receive current message, current message is repetition message;
If do not find the Message Record that keyword is identical in preset record sheet, then judge not receive current message, current message is non-duplicate message.
Thus, message consumer obtains the result of determination to current message.
Using warehousing system as message consumer, be illustrated.
Warehousing system receives order messages, revises adjustment accordingly to storage data.Order number in order messages, is the keyword of order messages.
Whether, after warehousing system receives order messages, warehousing system reads the order number of order messages, then search in preset record sheet and have keyword to be the order messages of this order number.
If find the order messages that keyword is identical in record sheet, then warehousing system judges to have received current order messages, and current order message is repetition message;
If do not find the order messages that keyword is identical in record sheet, then warehousing system judges not receive current order messages, and current order message is non-duplicate message.
If step S30 did not receive described message, then processed described message.
After the result of determination obtaining current message, if do not receive current message, also namely current message was non-duplicate message, and message consumer processes current message.
Concrete, as a kind of execution mode, first, message consumer's buffer memory current message, enters queue medium pending by current message.
Then, message consumer, according to the buffer memory order of the non-duplicate message obtained, reads the non-duplicate message obtained successively, carries out corresponding Business Processing, complete the process to current message from queue.
After processing current carrying out, message consumer deletes the current message in buffer memory, alleviates the storage burden of message consumer.
Using warehousing system as message consumer, be illustrated.
Warehousing system buffer memory current order message, enters queue by current order message medium pending.Then, warehousing system obtains order messages successively from queue, according to order messages, searches corresponding storage data, revises adjustment accordingly, the increase of such as storage data or minimizing to storage data.Then, then from queue, obtain next order messages, process accordingly.
In the present embodiment, message consumer according to preset record sheet, judges whether to receive current message, records the message that message consumer had received in record sheet after obtaining the message of server cluster propelling movement; If do not receive current message, then processed current message, and avoided that reprocessing is carried out to same message and take resource.The present embodiment achieves the filtration to repetition message, and message consumer only processes non-repetitive message, decreases resource occupation, and Message Processing efficiency is high, improves the validity of service.
Further, with reference to Fig. 2, message treatment method second embodiment of the present invention provides a kind of message treatment method, based on the embodiment shown in above-mentioned Fig. 1, also comprises after described step S20:
If step S40 received described message, then delete described message, and proceed to execution step S10.
After acquisition is to the result of determination of current message, if received current message, then current message was repetition message.
Concrete, as a kind of execution mode, first, message consumer deletes current message.
Then, message consumer obtains the lower a piece of news that server cluster pushes, and judges, obtain result of determination according to preset record sheet to lower a piece of news.
Then, according to the result of determination obtained, message consumer deletes repetition message, process non-duplicate message.
In the present embodiment, if received current message, then message consumer deleted current message, and obtained the lower a piece of news of server cluster propelling movement, processed accordingly.The present embodiment achieves the duplicate removal to the message received, and avoids, to the process of repetition message, decreasing resource occupation, and Message Processing efficiency is high, improves the validity of service.
Further, with reference to Fig. 3, message treatment method of the present invention 3rd embodiment provides a kind of message treatment method, based on the embodiment shown in above-mentioned Fig. 1, after described step S20, also comprises:
If step S50 did not receive described message, then in described record sheet, recorded described message, to upgrade described record sheet.
After the result of determination obtaining current message, if do not receive current message, also namely current message was non-duplicate message, and message consumer records current message, to upgrade record sheet.
Concrete, as a kind of execution mode, first, message consumer obtains the keyword of current message, the non-duplicate message that the sign that the keyword obtained can be unique is current.
Then, the keyword message obtained is reported in record sheet by message consumer, records the non-duplicate message that this obtains, for real-time update record sheet, obtains the record sheet dynamically updated.
Thus, message consumer obtains the record sheet after upgrading, and is judged by the record sheet after upgrading as preset record sheet to the follow-up message received.
In the present embodiment, if do not receive current message, then message consumer recorded the current message of acquisition in record sheet, to upgrade record sheet, for the judgement of subsequent message.This enforcement achieves dynamically updating of preset record sheet, and message consumer can judge the message received according to the record sheet that message is complete, obtains result of determination accurately, alleviates the processing load of message consumer, promotes treatment effeciency.
Further, with reference to Fig. 4, message treatment method of the present invention 4th embodiment provides a kind of message treatment method, based on the embodiment (the present embodiment is for Fig. 1) shown in above-mentioned Fig. 1,2 or 3, also comprises after described step S20:
The duplicate removal effect that step S60, basis are preset and/or processing speed, delete the record in described record sheet in preset time range at the timing node preset.
According to message consumer to the duplicate removal effect and/or the processing speed that receive message, the part record in message consumer's timing deletion record table.Message consumer by default regular time node and time range, after the timing node preset arrives, the Message Record in deletion record table in preset time range.
Concrete, as a kind of execution mode, message consumer according to the duplicate removal effect preset, setup times node and time range, the part record in deletion record table.
The message recorded in record sheet is more, message consumer can search more complete Message Record according to record sheet, obtain result of determination more accurately, remove more repetition message, the ratio that the repetition message quantity removed accounts for the repetition message sum received is higher, and message consumer's duplicate removal effect is better.
Therefore, message consumer is according to the duplicate removal target effect preset, and the mode delete number of times by increasing, expanding deletion scope reduces duplicate removal effect, also namely increases default timing node quantity, expands the time range preset; Promote duplicate removal effect by the mode that number of times is deleted in minimizing, reduce deletion scope, also namely reduce default timing node quantity, reduce default time range.
As another kind of execution mode, message consumer according to preset processing speed, setup times node and time range, the part record in deletion record table.
Because the Message Record in record sheet is in real-time update, Message Record is more, and need data to be processed more when message consumer searches Message Record, seek rate is slower, and processing speed is lower.
Therefore, message consumer is according to the processing speed target effect preset, and the mode delete number of times by increasing, expanding deletion scope improves processing speed, also namely increases default timing node quantity, expands the time range preset; Reduce processing speed by the mode that number of times is deleted in minimizing, reduce deletion scope, also namely reduce default timing node quantity, reduce default time range.
As another kind of execution mode, message consumer can according to the duplicate removal effect preset and processing speed, setup times node and time range, the part record in deletion record table.
The message recorded in record sheet is more, and message consumer can search more complete Message Record according to record sheet, obtains result of determination more accurately, removes more repetition message.But need data to be processed more when searching Message Record, seek rate is slower, and processing speed is lower.Therefore, the timing node preset and time range according to the actual treatment ability of the duplicate removal effect of message and message consumer, can set, make the processing speed of message consumer and duplicate removal effect reach the optimum state of coordination flexibly.
Such as, when the timing node preset is 12 of every day, before default time range is 48 hours of current time.Then message consumer is 12 of every day time, in deletion record table, take current time as timing node, the Message Record before 48 hours, also namely deletes the Message Record before two days.
In the present embodiment, message consumer according to preset duplicate removal effect and/or processing speed, preset timing node, the record in deletion record table in preset time range.The present embodiment effectively alleviates the burden that message consumer judges message, improves disposal ability and the treatment effeciency of message consumer, achieves the stability of message consumer's processing messages.
Further, with reference to Fig. 5, message treatment method of the present invention 5th embodiment provides a kind of message treatment method, based on the embodiment (the present embodiment is for Fig. 1) shown in above-mentioned Fig. 1,2 or 3, also comprises after described step S10:
S70, to described server set pocket transmission response, and obtain the next message that described server cluster pushes.
After the message receiving server cluster propelling movement, message consumer is to server set pocket transmission response, and announcement server cluster receives current message.
Concrete, as a kind of execution mode, after the message receiving server cluster propelling movement, message consumer is namely to server set pocket transmission response, and announcement server receives current message, and is processing.
Then, message consumer continues the next message that reception server cluster pushes.
In the present embodiment, message consumer, after server set pocket transmission response, obtains the next message that server cluster pushes.The present embodiment achieves the stability of Message Processing, makes message consumer in the peak period of message request, orderly reception and processing messages.
With reference to Fig. 6, message processing apparatus first embodiment of the present invention provides a kind of message processing apparatus, and described message processing apparatus comprises:
Acquisition module 100, obtains the message of server cluster propelling movement for message consumer.
The present invention is mainly used in the Message Processing pattern of the message producer, server cluster and message consumer.In the present embodiment, get one or more message producer cluster as the message producer, each message producer cluster can comprise one or more producer, the producer for the production of and send message; Get one or more message consumer's cluster as message consumer, each message consumer's cluster can comprise one or more consumer, and consumer is for the treatment of message.
In the message that the message producer generates, after being pushed to message consumer by server cluster, message consumer obtains the message that server cluster pushes and to go forward side by side row relax.
Concrete, as a kind of execution mode, acquisition module 100 is the message that pushes of reception server cluster successively, obtains the message that server cluster pushes.
Using the order of acquisition module 100 receipt message as follow-up order of carrying out Message Processing.
Judge module 200, for according to preset record sheet, judges whether to receive described message, records the message that described message consumer had received in described record sheet.
After acquisition module 100 obtains message, judge module 200 judges current message.
Concrete, as a kind of execution mode, first, judge module 200 obtains the keyword of current message.Wherein, the sign a piece of news that the keyword of message can be unique, the message that keyword is identical is repetition message, and the content of repetition message is identical.
Then, judge module 200 searches preset record sheet, searches the Message Record whether having keyword identical.Wherein, preset record sheet, with the form of Message Record, records the paid-in message of message consumer.Judge module 200, by being mated with the keyword of current message by the keyword in Message Record, is searched and whether is had the Message Record identical with current message keyword.
If find the Message Record that keyword is identical in preset record sheet, then judge to receive current message, current message is repetition message;
If do not find the Message Record that keyword is identical in preset record sheet, then judge not receive current message, current message is non-duplicate message.
Thus, judge module 200 obtains the result of determination to current message.
Using warehousing system as message consumer, be illustrated.
Warehousing system receives order messages, revises adjustment accordingly to storage data.Order number in order messages, is the keyword of order messages.
Whether, after the acquisition module 100 of warehousing system receives order messages, the judge module 200 of warehousing system reads the order number of order messages, then search in preset record sheet and have keyword to be the order messages of this order number.
If find the order messages that keyword is identical in record sheet, then the judge module 200 of warehousing system judges to receive current order messages, and current order message is repetition message;
If do not find the order messages that keyword is identical in record sheet, then the judge module 200 of warehousing system judges not receive current order messages, and current order message is non-duplicate message.
Processing module 300, if for not receiving described message, then process described message.
Obtain the result of determination of current message at judge module 200 after, if do not receive current message, also namely current message was non-duplicate message, and processing module 300 processes current message.
Concrete, as a kind of execution mode, first, processing module 300 buffer memory current message, enters queue medium pending by current message.
Then, processing module 300, according to the buffer memory order of the non-duplicate message obtained, reads the non-duplicate message obtained successively, carries out corresponding Business Processing, complete the process to current message from queue.
After processing current message, processing module 300 deletes the current message in buffer memory, alleviates the storage burden of message consumer.
Using warehousing system as message consumer, be illustrated.
Warehousing system processing module 300 buffer memory current order message, enters queue by current order message medium pending.Then, the order messages that warehousing system processing module 300 obtains successively from queue, according to order messages, searches corresponding storage data, revises adjustment accordingly, the increase of such as storage data or minimizing to storage data.
In the present embodiment, after acquisition module 100 obtains the message of server cluster propelling movement, judge module 200, according to preset record sheet, judges whether to receive current message, records the message that message consumer had received in record sheet; If do not receive current message, then processing module 300 processed current message, avoided carrying out reprocessing to same message and took resource.The present embodiment achieves the filtration to repetition message, and message consumer only processes non-repetitive message, decreases resource occupation, and Message Processing efficiency is high, improves the validity of service.
Further, with reference to Fig. 7, message processing apparatus second embodiment of the present invention provides a kind of message processing apparatus, and based on the embodiment shown in above-mentioned Fig. 6, described message processing apparatus also comprises:
Duplicate removal module 400, if for receiving described message, then delete described message; Described acquisition module 100, also obtains the next message of server cluster propelling movement for message consumer.
After the result of determination that judge module 200 obtains current message, if received current message, then current message was repetition message.
Concrete, as a kind of execution mode, first, duplicate removal module 400 deletes current message.
Then, acquisition module 100 obtains the lower a piece of news that server cluster pushes, and judge module 200 judges lower a piece of news according to preset record sheet, obtains result of determination.
Then, according to the result of determination obtained, duplicate removal module 400 deletes repetition message, and processing module 300 processes non-duplicate message.
In the present embodiment, if received current message, then duplicate removal module 400 deleted current message, and obtained the lower a piece of news of server cluster propelling movement, processed accordingly.The present embodiment achieves the duplicate removal to the message received, and avoids, to the process of repetition message, decreasing resource occupation, and Message Processing efficiency is high, improves the validity of service.
Further, with reference to Fig. 8, message processing apparatus of the present invention 3rd embodiment provides a kind of message processing apparatus, and based on the embodiment shown in above-mentioned Fig. 6, described message processing apparatus also comprises:
Logging modle 500, if for not receiving described message, then record described message in described record sheet, to upgrade described record sheet.
Obtain the result of determination of current message at judge module 200 after, if do not receive current message, also namely current message was after non-duplicate message, and logging modle 500 records current message, to upgrade record sheet.
Concrete, as a kind of execution mode, first, logging modle 500 obtains the keyword of current message, the non-duplicate message that the sign that the keyword obtained can be unique is current.
Then, the keyword message obtained is reported in record sheet by logging modle 500, records the non-duplicate message that this obtains, for real-time update record sheet, obtains the record sheet dynamically updated.
Thus, logging modle 500 obtains the record sheet after upgrading, and is judged by the record sheet after upgrading as preset record sheet to the follow-up message received.
In the present embodiment, if do not receive current message, then logging modle 500 recorded current message in record sheet, to upgrade record sheet, for the judgement of subsequent message.This enforcement achieves dynamically updating of preset record sheet, and message consumer can judge the message received according to the record sheet that message is complete, obtains result of determination accurately, alleviates the processing load of message consumer, promotes treatment effeciency.
Further, with reference to Fig. 9, message processing apparatus of the present invention 4th embodiment provides a kind of message processing apparatus, and based on the embodiment (the present embodiment is for Fig. 6) shown in above-mentioned Fig. 6,7 or 8, described message processing apparatus also comprises:
Removing module 600, for according to the duplicate removal effect preset and/or processing speed, deletes the record in described record sheet in preset time range at the timing node preset.
According to the duplicate removal effect and/or the processing speed that receive message, the part record in removing module 600 timing deletion record table.Removing module 600 by default regular time node and time range, after the timing node preset arrives, the Message Record in deletion record table in preset time range.
Concrete, as a kind of execution mode, removing module 600 according to the duplicate removal effect preset, setup times node and time range, the part record in deletion record table.
The message recorded in record sheet is more, judge module 200 can search more complete Message Record according to record sheet, obtain result of determination more accurately, remove more repetition message, the ratio that the repetition message quantity removed accounts for the repetition message sum received is higher, and message consumer's duplicate removal effect is better.
Therefore, removing module 600 is according to the duplicate removal target effect preset, and the mode delete number of times by increasing, expanding deletion scope reduces duplicate removal effect, also namely increases default timing node quantity, expands the time range preset; Promote duplicate removal effect by the mode that number of times is deleted in minimizing, reduce deletion scope, also namely reduce default timing node quantity, reduce default time range.
As another kind of execution mode, removing module 600 according to preset processing speed, setup times node and time range, the part record in deletion record table.
Because the Message Record in record sheet is in real-time update, Message Record is more, and need data to be processed more when judge module 200 searches Message Record, seek rate is slower, and processing speed is lower.
Therefore, removing module 600 is according to the processing speed target effect preset, and the mode delete number of times by increasing, expanding deletion scope improves processing speed, also namely increases default timing node quantity, expands the time range preset; Reduce processing speed by the mode that number of times is deleted in minimizing, reduce deletion scope, also namely reduce default timing node quantity, reduce default time range.
As another kind of execution mode, removing module 600 can according to the duplicate removal effect preset and processing speed, setup times node and time range, the part record in deletion record table.
The message recorded in record sheet is more, and judge module 200 can search more complete Message Record according to record sheet, obtains result of determination more accurately, removes more repetition message.But need data to be processed more when searching Message Record, seek rate is slower, and processing speed is lower.Therefore, the timing node preset and time range according to the actual treatment ability of the duplicate removal effect of message and message consumer, can set, make the processing speed of judge module 200 and duplicate removal effect reach the optimum state of coordination flexibly.
Such as, when the timing node preset is 12 of every day, before default time range is 48 hours of current time.Then removing module 600 is 12 of every day time, in deletion record table, take current time as timing node, the Message Record before 48 hours, also namely deletes the Message Record before two days.
In the present embodiment, removing module 600 according to preset duplicate removal effect and/or processing speed, preset timing node, the record in deletion record table in preset time range.The present embodiment effectively alleviates the burden that duplicate removal module judges message, improves disposal ability and the treatment effeciency of message consumer, achieves the stability of message consumer's processing messages.
Further, with reference to Figure 10, message processing apparatus of the present invention 5th embodiment provides a kind of message processing apparatus, and based on the embodiment (the present embodiment is for Fig. 6) shown in above-mentioned Fig. 6,7 or 8, described message processing apparatus also comprises:
Responder module 700, for replying to described server set pocket transmission.
Receive the message of server cluster propelling movement at acquisition module 100 after, responder module 700 is to server set pocket transmission response, and announcement server cluster receives current message.
Concrete, as a kind of execution mode, receive the message of server cluster propelling movement at acquisition module 100 after, responder module 700 is namely to server set pocket transmission response, and announcement server receives current message, and is processing.
Then, the next message of reception server cluster propelling movement is continued by acquisition module 100.
In the present embodiment, responder module 700 is after server set pocket transmission response, and acquisition module 100 obtains the next message that server cluster pushes.The present embodiment achieves the stability of Message Processing, makes message consumer in the peak period of message request, orderly reception and processing messages.
With reference to Figure 11, message handling system of the present invention provides a kind of message handling system, and described message handling system comprises message producer cluster A, server cluster B and message consumer's cluster C, wherein:
Described message producer cluster A, gives described server cluster for the production of information and sending.
In the present embodiment, message producer cluster A comprises one or more message producer cluster, and each message producer cluster can comprise one or more producer, the producer for the production of and send message.Wherein, the message that the producer generates includes the keyword that uniquely can characterize current message, and the message that keyword is identical is repetition message, and the content of repetition message is identical.
Such as sell official website, user uses and sells official website generation sales order, and sell official website as message producer cluster A, the order messages producer generates order messages, and sends to server cluster.Order messages includes keyword order number, for unique sign current message.
Described server cluster B, for receiving the message that described message producer cluster sends, and is pushed to described message consumer's cluster.
In the present embodiment, server cluster B comprises one or more server, for accepting the message that message producer cluster sends respectively, and message is pushed to corresponding message consumer respectively.
Server cluster B can process the transmitting-receiving of a large amount of message, alleviates the burden of message consumer during message request peak, as a kind of execution mode, RabbitMQ (MessageQueue, message queue) can be used as server cluster.
Described message consumer's cluster C, comprises acquisition module 100, judge module 200, processing module 300, duplicate removal module 400, logging modle 500, removing module 600 and responder module 700.
In the present embodiment, message consumer's cluster C comprises one or more message consumer's cluster, and each message consumer's cluster can comprise one or more consumer, and consumer is for the treatment of message.
The message that each consumer's difference reception server cluster B pushes, then judges the message received according to the record sheet preset, the message that deletion received, processes the message do not received.Each consumer records the non-duplicate information updating record sheet received, for the judgement of message; Message Record in timing deletion record table, improves the treatment effeciency judged message; Answering server cluster B, the message that orderly reception server cluster B pushes.
Concrete, with reference to Figure 12, as a kind of execution mode, consumer can use filter module C1 receipt message and carry out duplicate removal to message, the non-duplicate message that local message queue cache module C2 buffer memory obtains, Service Processing Module C3 extracts non-duplicate message and processes from local message queue cache module C2.
Wherein, filter module C1 can adopt database Redis, provides the unrepeatable data structure of supported data in Redis, as: set, zset (sortedset).Local message queue cache module C2 can adopt instrument class libraries java.util.
In the present embodiment, message handling system comprises message producer cluster A, server cluster B and message consumer's cluster C.Message producer cluster A production information and sending is to server cluster B.The server cluster B receipt message producer collects the message of A pocket transmission, and is pushed to message consumer's cluster C.The message that message consumer's cluster C, reception server cluster B push, and remove repetition message, non-duplicate message is processed accordingly.The present embodiment achieves the lifting of Message Processing efficiency, decreases resource occupation, and Message Processing efficiency is high, improves the validity of service.
Further, with reference to Figure 13, message handling system second of the present invention is implemented to provide a kind of message handling system, and based on the embodiment shown in above-mentioned Figure 11, described server cluster B comprises one or more server D, wherein:
Described server D comprises switch module D1 and Queue module D2.
The message that server D receipt message producer cluster A sends, and be pushed to message consumer's cluster C.
Concrete, as a kind of execution mode, server D comprises switch module D1 and Queue module D2.Each server D all can comprise one or more switch module D1.Each switch module D1 all may correspond to one or more Queue module D2, and message is sent to corresponding Queue module D2 by switch module D1.
Wherein, the producer in switch module D1 and message producer cluster A pre-sets corresponding relation, and switch module D1 receives the message that the corresponding producer sends.The message that the producer sends carries route key, and route key is also consumer for characterizing the receiving terminal of current message.
Consumer in Queue module D2 and message consumer's cluster C pre-sets corresponding relation and binds, and each Queue module D2 all has the consumer of one_to_one corresponding binding.Queue module D2 pushes to the consumer of binding the message received.
Queue module D2, according to the consumer of binding, receives the message of carrying coupling route key.The route key mated with Queue module D2, characterizes the consumer that Queue module D2 binds.
Described switch module D1, for receiving the message that described message producer cluster A sends, sends to the Queue module D2 with the described match messages received by the described message received.
Switch module D1 is after the message receiving corresponding producer's transmission, concrete, and as a kind of execution mode, first, switch module D1 obtains the route key that current message carries.
Then, current message, according to route key, is sent to the Queue module D2 mated with current route key by switch module D1.
Described Queue module D2, for receiving the message that described switch module D1 sends, the described message received is pushed to described message consumer's cluster C successively by the rule preset, after receiving the response of described message consumer's cluster C, deletes the corresponding message of described response.
Queue module D2 desampler module D1, the message then will received, according to order when receiving, comes in queue successively by the message received.The message received is come queue rearmost end by Queue module D2, pushes queue message foremost to the consumer of binding.The message number that the queue of Queue module D2 can enter has default maximum, and when after the maximum that the message number in queue reaches default, Queue module D2 no longer receives new information, receives new information again when waiting list produces room.
Message push is given the consumer of binding by Queue module D2, after the consumer of binding receives message, sends response, inform that Queue module D2 receives current message, and processing accordingly to Queue module D2.
After Queue module D2 receives the response of the consumer of binding, delete the message corresponding to response, then to push in current queue message foremost to the consumer of binding.
As a kind of execution mode, the realization of Queue module D2 can use blocks queue LinkedBlockingQueue.
Be illustrated with reference to Figure 14, message producer cluster comprises message producer cluster A1, message producer cluster A2 and message producer cluster A3, server cluster is RabbitMQ cluster, and RabbitMQ cluster comprises server B 1, and message consumer's cluster comprises message consumer's cluster C1.
Wherein, RabbitMQ cluster comprises switch module B11, switch module B12 and switch module B13.The corresponding Queue module B11-1 and Queue module B11-2 of switch module B11; The corresponding Queue module B12-1 and Queue module B12-2 of switch module B12; The corresponding Queue module B13-1 and Queue module B13-2 of switch module B13.Individual queue module all binds the consumer in message consumer's cluster C1.
Concrete, A1 is corresponding with switch module B11 for message producer cluster, and message producer cluster A1 production information and sending is to switch module B11.After switch module B11 receives message, according to the route key that message is carried, message is sent to Queue module B11-1 or the Queue module B11-2 of coupling.The message received is entered queue by Queue module B11-1 or Queue module B11-2, is pushed to the consumer of binding successively.
A2 is corresponding with switch module B12 for message producer cluster, and message producer cluster A2 production information and sending is to switch module B12.After switch module B12 receives message, according to the route key that message is carried, message is sent to Queue module B12-1 or the Queue module B12-2 of coupling.The message received is entered queue by Queue module B12-1 or Queue module B12-2, is pushed to the consumer of binding successively.
A3 is corresponding with switch module B13 for message producer cluster, and message producer cluster A3 production information and sending is to switch module B13.After switch module B13 receives message, according to the route key that message is carried, message is sent to Queue module B13-1 or the Queue module B13-2 of coupling.The message received is entered queue by Queue module B13-1 or Queue module B13-2, is pushed to the consumer of binding successively.
In the present embodiment, server cluster B comprises one or more server D, and each server D comprises switch module D1 and Queue module D2; Switch module D1, for the message that receipt message producer cluster A sends, sends to the Queue module D2 with the match messages received by the message received; Queue module D2, for the message that desampler module D1 sends, is pushed to message consumer's cluster C by the message received by the rule preset successively, after receiving the response of message consumer's cluster C, deletes the corresponding message of response.In the present embodiment, the message received is distributed by server cluster, and is pushed to corresponding consumer, alleviates the burden of message consumer's cluster, maintain the stability that message handling system carries out Message Processing, improve the Message Processing efficiency of message handling system.
These are only the preferred embodiments of the present invention; not thereby the scope of the claims of the present invention is limited; every utilize specification of the present invention and accompanying drawing content to do equivalent structure or equivalent flow process conversion; or be directly or indirectly used in other relevant technical fields, be all in like manner included in scope of patent protection of the present invention.
Claims (12)
1. a message treatment method, is characterized in that, described message treatment method comprises the following steps:
Message consumer obtains the message that server cluster pushes;
According to preset record sheet, judge whether to receive described message, in described record sheet, record the message that described message consumer had received;
If do not receive described message, then processed described message.
2. message treatment method as claimed in claim 1, is characterized in that, described according to preset record sheet, judges whether to receive described message, after recording the step of the message that described message consumer had received, also comprises in described record sheet:
If received described message, then deleted described message, and proceeded to execution step: message consumer obtained the message that server cluster pushes.
3. message treatment method as claimed in claim 1, is characterized in that, described according to preset record sheet, judges whether to receive described message, after recording the step of the message that described message consumer had received, also comprises in described record sheet:
If do not receive described message, then in described record sheet, recorded described message, to upgrade described record sheet.
4. the message treatment method as described in claim 1,2 or 3, is characterized in that, does not receive described message if described, then, after processing the step of described message, also comprise:
According to the duplicate removal effect preset and/or processing speed, delete the record in described record sheet in preset time range at the timing node preset.
5. the message treatment method as described in claim 1,2 or 3, is characterized in that, described message consumer also comprises after obtaining the step of the message that server cluster pushes:
To described server set pocket transmission response, and obtain the next message of described server cluster propelling movement.
6. a message processing apparatus, is characterized in that, described message processing apparatus comprises:
Acquisition module, obtains the message of server cluster propelling movement for message consumer;
Judge module, for according to preset record sheet, judges whether to receive described message, records the message that described message consumer had received in described record sheet;
Processing module, if for not receiving described message, then process described message.
7. message processing apparatus as claimed in claim 6, it is characterized in that, described message processing apparatus also comprises:
Duplicate removal module, if for receiving described message, then delete described message;
Described acquisition module, also obtains the next message of server cluster propelling movement for message consumer.
8. message processing apparatus as claimed in claim 6, it is characterized in that, described message processing apparatus also comprises:
Update module, if for not receiving described message, then recorded described non-duplicate message, to upgrade described record sheet in described record sheet.
9. the message processing apparatus as described in claim 6,7 or 8, is characterized in that, described message processing apparatus also comprises:
Removing module, for according to the duplicate removal effect preset and/or processing speed, deletes the record in described record sheet in preset time range at the timing node preset.
10. the message processing apparatus as described in claim 6,7 or 8, is characterized in that, described message processing apparatus also comprises:
Responder module, for replying to described server set pocket transmission.
11. 1 kinds of message handling systems, is characterized in that, described message handling system comprises message producer cluster, server cluster and message consumer's cluster, wherein:
Described message producer cluster, gives described server cluster for the production of information and sending;
Described server cluster, for receiving the message that described message producer cluster sends, and is pushed to described message consumer's cluster;
Described message consumer's cluster, comprises the message processing apparatus according to any one of claim 6-10.
12. message handling systems as claimed in claim 11, it is characterized in that, described server cluster comprises one or more server, wherein:
Described server comprises switch module and Queue module;
Described switch module, for receiving the message that described message producer cluster sends, sends to the Queue module with the described match messages received by the described message received;
Described Queue module, for receiving the message that described switch module sends, being pushed to described message consumer's cluster by the described message received successively by the rule preset, after receiving the response of described message consumer's cluster, deleting the corresponding message of described response.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510646199.XA CN105227662A (en) | 2015-09-30 | 2015-09-30 | Message treatment method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510646199.XA CN105227662A (en) | 2015-09-30 | 2015-09-30 | Message treatment method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105227662A true CN105227662A (en) | 2016-01-06 |
Family
ID=54996332
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510646199.XA Pending CN105227662A (en) | 2015-09-30 | 2015-09-30 | Message treatment method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105227662A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106170013A (en) * | 2016-06-29 | 2016-11-30 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of Kafka message uniqueness method based on Redis |
| CN106372204A (en) * | 2016-08-31 | 2017-02-01 | 北京小米移动软件有限公司 | Push message processing method and device |
| CN106789741A (en) * | 2016-12-26 | 2017-05-31 | 北京奇虎科技有限公司 | The consuming method and device of message queue |
| CN107222530A (en) * | 2017-05-23 | 2017-09-29 | 努比亚技术有限公司 | Service asynchronous exchange method, equipment, system and computer-readable recording medium |
| CN108984325A (en) * | 2018-07-20 | 2018-12-11 | 北京北信源信息安全技术有限公司 | Message queue consuming method and device |
| CN109582470A (en) * | 2017-09-28 | 2019-04-05 | 北京国双科技有限公司 | A kind of data processing method and data processing equipment |
| CN112637046A (en) * | 2020-12-18 | 2021-04-09 | 北京房天下嘉居网络技术有限公司 | Message pushing method and routing device |
| CN113296976A (en) * | 2021-02-10 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Message processing method, message processing device, electronic equipment, storage medium and program product |
| CN113778321A (en) * | 2021-01-15 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Message processing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999527A (en) * | 2011-09-16 | 2013-03-27 | 腾讯科技（深圳）有限公司 | Dynamic information pushing method and system in SNS (social networking services) network |
| JP2013175072A (en) * | 2012-02-27 | 2013-09-05 | Hitachi Ltd | Test apparatus of program and test system of program |
| CN103780677A (en) * | 2013-12-24 | 2014-05-07 | 西安酷派软件科技有限公司 | Method for performing classified information push and system thereof |
| CN104252665A (en) * | 2014-09-16 | 2014-12-31 | 国家海洋信息中心 | Method and system for managing marine environment monitoring data |
| CN104636477A (en) * | 2015-02-15 | 2015-05-20 | 山东卓创资讯集团有限公司 | Push list duplicate removal method before information push |
| CN104754036A (en) * | 2015-03-06 | 2015-07-01 | 合一信息技术（北京）有限公司 | Message processing system and processing method based on kafka |
-
2015
- 2015-09-30 CN CN201510646199.XA patent/CN105227662A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999527A (en) * | 2011-09-16 | 2013-03-27 | 腾讯科技（深圳）有限公司 | Dynamic information pushing method and system in SNS (social networking services) network |
| JP2013175072A (en) * | 2012-02-27 | 2013-09-05 | Hitachi Ltd | Test apparatus of program and test system of program |
| CN103780677A (en) * | 2013-12-24 | 2014-05-07 | 西安酷派软件科技有限公司 | Method for performing classified information push and system thereof |
| CN104252665A (en) * | 2014-09-16 | 2014-12-31 | 国家海洋信息中心 | Method and system for managing marine environment monitoring data |
| CN104636477A (en) * | 2015-02-15 | 2015-05-20 | 山东卓创资讯集团有限公司 | Push list duplicate removal method before information push |
| CN104754036A (en) * | 2015-03-06 | 2015-07-01 | 合一信息技术（北京）有限公司 | Message processing system and processing method based on kafka |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106170013A (en) * | 2016-06-29 | 2016-11-30 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of Kafka message uniqueness method based on Redis |
| CN106372204A (en) * | 2016-08-31 | 2017-02-01 | 北京小米移动软件有限公司 | Push message processing method and device |
| CN106789741A (en) * | 2016-12-26 | 2017-05-31 | 北京奇虎科技有限公司 | The consuming method and device of message queue |
| CN107222530A (en) * | 2017-05-23 | 2017-09-29 | 努比亚技术有限公司 | Service asynchronous exchange method, equipment, system and computer-readable recording medium |
| CN109582470A (en) * | 2017-09-28 | 2019-04-05 | 北京国双科技有限公司 | A kind of data processing method and data processing equipment |
| CN108984325A (en) * | 2018-07-20 | 2018-12-11 | 北京北信源信息安全技术有限公司 | Message queue consuming method and device |
| CN112637046A (en) * | 2020-12-18 | 2021-04-09 | 北京房天下嘉居网络技术有限公司 | Message pushing method and routing device |
| CN113778321A (en) * | 2021-01-15 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Message processing method and device |
| CN113296976A (en) * | 2021-02-10 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Message processing method, message processing device, electronic equipment, storage medium and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105227662A (en) | Message treatment method, device and system | |
| CN110413650B (en) | Method, device, equipment and storage medium for processing service data | |
| CN103780679A (en) | Long time delay remote invocation method based on HTTP protocol | |
| CN101646140B (en) | Message log processing method and system | |
| CN111221469B (en) | Method, device and system for synchronizing cache data | |
| CN111258978A (en) | Data storage method | |
| RU2545999C2 (en) | Method, apparatus and mobile broadcast business management system for transmitting information in data form | |
| CN109241099B (en) | Data query method and terminal equipment | |
| CN113886494A (en) | Message storage method, device, equipment and computer readable medium for instant messaging | |
| CN104636477A (en) | Push list duplicate removal method before information push | |
| CN111597259B (en) | Data storage system, method, device, electronic equipment and storage medium | |
| CN107870982A (en) | Data processing method, system and computer-readable recording medium | |
| CN105468346A (en) | Timing event management method, storage method, execution method and corresponding apparatus | |
| CN114244752A (en) | Flow statistical method, device and equipment | |
| CN105095224A (en) | Method, apparatus and system for carrying out OLAP analysis in mobile communication network | |
| CN111190861B (en) | Hot spot file management method, server and computer readable storage medium | |
| CN109976896B (en) | Service re-ranking processing method and device | |
| CN102143440A (en) | Method and device for processing short message service | |
| CN104065617A (en) | Harassing-email processing method, device and system | |
| CN105868057A (en) | Data processing method and device as well as mobile terminal | |
| CN114063931A (en) | Data storage method based on big data | |
| CN101494833B (en) | Method, device and system for sending network message | |
| CN111061719B (en) | Data collection method, device, equipment and storage medium | |
| CN116542668A (en) | Block chain-based data processing method, equipment and readable storage medium | |
| CN115794876A (en) | Fragment processing method, device, equipment and storage medium for service data packet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160106 |
|
| RJ01 | Rejection of invention patent application after publication |