CN102955717A - Message management equipment and method in distributed message processing system - Google Patents
Message management equipment and method in distributed message processing system Download PDFInfo
- Publication number
- CN102955717A CN102955717A CN2012104363857A CN201210436385A CN102955717A CN 102955717 A CN102955717 A CN 102955717A CN 2012104363857 A CN2012104363857 A CN 2012104363857A CN 201210436385 A CN201210436385 A CN 201210436385A CN 102955717 A CN102955717 A CN 102955717A
- Authority
- CN
- China
- Prior art keywords
- message
- information
- partition holding
- processing device
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses message management equipment and method in a distributed message processing system. The message management method includes the steps of: receiving a message processing request sent by the message processing equipment, wherein the message processing request comprises at least the subject of a message requested to be processed; inquiring storage partition information corresponding to the subject in the message processing request, and sending the inquired storage partition information to the corresponding message processing equipment; processing a message location inquiring request from the message processing equipment, inquiring the location information of the message currently requested to be processed in a storage partition corresponding to the storage partition in the message location inquiring request, and sending the location information to the corresponding message processing equipment; and processing a message location update request from the message processing equipment, updating the location information of the message currently requested to be processed in a storage partition corresponding to the storage partition in the message location update request on the basis of the message size in the message location update request.
Description
Technical field
The present invention relates to the Message Processing field, be specifically related to a kind of message management equipment and method in the distributed message disposal system.
Background technology
The message system that is used at present message (such as daily record) processing has a lot, and popular is distributed information system.The concrete form of distributed information system has a lot, such as Spread, Kafka etc.
The below only describes as an example of a kind of distributed information system Kafka commonly used example.
Kafka is the distributed information system for log processing, and is large mainly for the daily record data capacity, but is not very high business to reliability requirement.Daily record data in these systems mainly comprises user behavior, for example, login, browse, click, share and like etc., also comprise system's running log, for example, the running status of CPU (Central Processing Unit, central processing unit), internal memory, disk, network, system and process etc.
Fig. 1 is Kafka system architecture synoptic diagram in the correlation technique.This Kafka system comprises following four roles: Producer (message producer, be called for short PD) 11, Broker (being called for short BK) 12, Consumer (Message Processing person, be called for short CS) 13 and Zookeeper (supvr, be called for short ZK) 14, each role can be a plurality of.Consumer 13 is responsible for processing messages in the Kafka system; Producer 11 is responsible for producing message in the Kafka system; Broker12 is the place that is used for depositing message in the Kafka system, is responsible for receiving the message that Producer 11 sends, and when Consumer 13 has request, these message is returned to Consumer13; ZooKeeper 14 is distributed application program coordination services of an open source code, comprise a simple primitive collection, be the significant components of Hadoop and Hbase, Distributed Application can realize such as functions such as unified naming service, configuration management, distributed lock service, cluster managements with it.Producer 11 sends a message to Broker12, and the message persistent storage is on Broker12, and Consumer 13 obtains message from Broker12 again and processes.Zookeeper 14 is used for storing Producer 11, and Consumer 13, with some status informations of Broker12.
The Kafka system is the mode that adopts pull (drawing) to the processing of message, obtains message to be processed by Consumer 13 from Broker 12 at every turn.The position Offset of the message that success is processed is preserved by Consumer13.
In above-mentioned message process mode, Consumer 13 needs own position Offset to the successful message of processing to manage, preserve, Offset generally is kept on the internal memory or local disk of the message processing device under the Consumer 13, stop or after the machine disk broke down, the Offset that before preserves had just lost when message processing device is unexpected.After restarting message processing device, the Offset that will preserve from the last time or the consumption that starts anew, Consumer 13 can't be right after the position continuation of the message of successfully processing last time message is processed.
Summary of the invention
In view of the above problems, the present invention has been proposed in order to a kind of message management equipment and method in the distributed message disposal system that overcomes the problems referred to above or address the above problem at least in part is provided.
According to one aspect of the present invention, a kind of message management equipment in the distributed message disposal system is provided, this distributed message disposal system also comprises one or more information storaging apparatus and one or more message processing device, wherein every message has corresponding theme, information storaging apparatus is that one or more partition holding set up in corresponding theme, and according to the theme of message with message stores in one of partition holding of corresponding theme, message management equipment comprises: message distribution status register, the information that is configured to store at least relevant partition holding in each information storaging apparatus, and the positional information of the current message to be processed of relevant information treatment facility in each partition holding; Transmission interface is configured to the Message Processing request that the receipt message treatment facility sends, and the Message Processing request comprises the theme of the processing messages of asking at least; The subregion state requestor is configured to the inquiry partition holding information corresponding with the theme in the Message Processing request in the message distribution status register, and via transmission interface the partition holding information that inquires is sent to the corresponding message treatment facility; And message location manager, be configured to process receive by transmission interface, from the message position query requests of message processing device, from the message distribution status register, obtain the current of the partition holding corresponding with the partition holding in this message position query requests and want the positional information of processing messages and send to the corresponding message treatment facility via transmission interface; And be configured to process receive by transmission interface, from the message position updating request of message processing device, based on the message size in the message position updating request, in the updating message distribution storer, the current positional information of wanting processing messages of the partition holding corresponding with the partition holding in the message position updating request.
Optionally, also store the size of the each message to be dealt with of message processing device in the message distribution status register; The subregion state requestor also is configured to the size of inquiry corresponding message treatment facility message to be dealt with in the message distribution status register, and sends to the corresponding message treatment facility by transmission interface.
According to another aspect of the present invention, a kind of information management method also is provided, the message management equipment that the method is suitable in the distributed message disposal system is carried out, this distributed message disposal system also comprises one or more information storaging apparatus and one or more message processing device, wherein every message has corresponding theme, information storaging apparatus is that one or more partition holding set up in corresponding theme, and according to the theme of message with message stores in one of partition holding of corresponding theme, information management method comprises: the Message Processing request that the receipt message treatment facility sends, and the Message Processing request comprises the theme of the processing messages of asking at least; Inquire about the partition holding information corresponding with the theme in the Message Processing request, and the partition holding information that inquires sent to the corresponding message treatment facility, wherein, store at least in each information storaging apparatus the positional information of the current message to be processed of relevant information treatment facility in the information of relevant partition holding and each partition holding in the message management equipment; And processing is inquired about partition holding current corresponding with the partition holding in this message position query requests and is wanted the positional information of processing messages and send to the corresponding message treatment facility from the message position query requests of message processing device; And processing based on the message size in the message position updating request, is upgraded the current positional information of wanting processing messages of the partition holding corresponding with the partition holding in the message position updating request from the message position updating request of message processing device.
Optionally, above-mentioned information management method also comprises: the size of inquiry corresponding message treatment facility message to be dealt with also sends to the corresponding message treatment facility, wherein also stores the size of the each message to be dealt with of message processing device in the message management equipment.
Technical scheme of the present invention stores in each information storaging apparatus the information of relevant partition holding at message management equipment, and the positional information of the current message to be processed of relevant information treatment facility in each partition holding, when message processing device obtains message to be processed, the partition holding information of message on information storaging apparatus under the theme that only needs to process from message management equipment query message processing device, and message processing device is in the positional information of the current message to be processed of corresponding partition holding, can send a request message and obtain message from information storaging apparatus and process, like this, even message processing device breaks down, after message processing device recovers normally, the position acquisition message that still can be right after success last time processing messages is processed, and has avoided re-treatment or the omission of message.
Above-mentioned explanation only is the general introduction of technical solution of the present invention, for can clearer understanding technological means of the present invention, and can be implemented according to the content of instructions, and for above and other objects of the present invention, feature and advantage can be become apparent, below especially exemplified by the specific embodiment of the present invention.
Description of drawings
By reading hereinafter detailed description of the preferred embodiment, various other advantage and benefits will become cheer and bright for those of ordinary skills.Accompanying drawing only is used for the purpose of preferred implementation is shown, and does not think limitation of the present invention.And in whole accompanying drawing, represent identical parts with identical reference symbol.In the accompanying drawings:
Fig. 1 shows a kind of Kafka system architecture synoptic diagram;
Fig. 2 shows the according to an embodiment of the invention synoptic diagram of distributed message disposal system;
Fig. 3 shows the according to an embodiment of the invention synoptic diagram of information storaging apparatus storage inside structure;
Fig. 4 shows the according to an embodiment of the invention process flow diagram of information management method; And
Fig. 5 shows the according to an embodiment of the invention process flow diagram of message treatment method.
Embodiment
Exemplary embodiment of the present disclosure is described below with reference to accompanying drawings in more detail.Although shown exemplary embodiment of the present disclosure in the accompanying drawing, yet should be appreciated that and to realize the disclosure and the embodiment that should do not set forth limits here with various forms.On the contrary, it is in order to understand the disclosure more thoroughly that these embodiment are provided, and can with the scope of the present disclosure complete convey to those skilled in the art.
Fig. 2 shows the according to an embodiment of the invention synoptic diagram of distributed message disposal system.As shown in the figure, this distributed message disposal system comprises that message produces equipment 100, information storaging apparatus 200, message processing device 300 and message management equipment 400.Message produces equipment 100, information storaging apparatus 200 and message processing device 300 respectively can be for one or more, only exemplaryly among Fig. 2 provided two message and produced equipment 100, two information storaging apparatus 200, and a message processing device 300 and a message management equipment 400, it will be understood by those skilled in the art that the embodiment of the invention produces the not restriction of number of equipment 100, information storaging apparatus 200, message processing device 300 and message management equipment 400 to message.
It can be that the network equipment, server, computing machine or other can produce any equipment of message that message produces equipment 100, and the message here can be the data of any kind of information, such as daily record data.Every message has corresponding theme, can be certain concrete user behavior such as this theme, as login behavior, browse behavior, the click behavior, share behavior or like behavior etc., and for example this theme can be certain concrete system's running log, such as CPU, internal memory, disk, network, system and running state of a process etc.For example, in one embodiment, it is the webserver that message produces equipment 100, and when the user browsed certain Webpage of this webserver, this webserver produced one thereupon and themes as the message of browsing this Webpage.
After message produces equipment 100 generation message, the message that produces is sent to information storaging apparatus 200.When message produces equipment 100 to information storaging apparatus 200 transmission message, can send at random, also can send according to the call back function of user's appointment.It is as follows with the code sample that the message that produces sends to information storaging apparatus 200 that message produces equipment 100:
$zkCluster＝′localhost:2181′；
$msg＝array′message?content′)；
$topic＝′test′；
$Producer＝new?Kafka_Producer($zkCluster)；
$Producer-＞send(array($msg)，$topic′)；
The code that it will be understood by those skilled in the art that above pass-along message only is an example, and any message that message is produced equipment 100 generations that can realize sends to the code realization of information storaging apparatus 200 all in protection scope of the present invention.
In order better to introduce the storage inside structure of information storaging apparatus 200, please be simultaneously referring to Fig. 3, it is the synoptic diagram of information storaging apparatus 200 storage inside structures according to an embodiment of the invention.In Fig. 3, show the first information storaging apparatus 2001, the second information storaging apparatus 2002 and the 3rd information storaging apparatus 2003, and the message stores path of each information storaging apparatus inside.In addition, also show the first message generation equipment 1001 and the second message generation equipment 1002 that carry out information interaction with these information storaging apparatus, and the first message processing device 3001 and the second message processing device 3002.
The first message produces equipment 1001 and the second message produces the message that themes as Topicl and Topic2 that equipment 1002 produces, and can be sent to the first information storaging apparatus 2001, the second information storaging apparatus 2002 and the 3rd information storaging apparatus 2003 of the message of the message that can be used for storing Topicl and Topic2.
Because the message based theme of distributed message disposal system (Topic) carries out message management, thus corresponding, in information storaging apparatus, also be based on Topic and store.The message of each Topic can be stored in one or more partition holdings (Partition), and when being stored in a plurality of partition holding, these a plurality of partition holdings can be positioned on the information storaging apparatus, also can be positioned on a plurality of information storaging apparatus.In Fig. 3, the message stores that themes as Topicl partition holding partl and partition holding part2 under the Topicl in the partition holding partl under the Topicl and partition holding part2, the second information storaging apparatus 2002 in the first information storaging apparatus 2001, and the partition holding partl under the Topicl in the 3rd information storaging apparatus 2003.The message stores that themes as Topic2 partition holding partl under the Topic2 in the partition holding partl under the Topic2, the second information storaging apparatus 2002 in the first information storaging apparatus 2001, and the partition holding partl under the Topic2 in the 3rd information storaging apparatus 2003.Wherein each partition holding has multiple messages, and every message has different positions with respect to the partition holding head.For example, according to one embodiment of present invention, every message all has message id, and message id is determined by its logical place, namely can be directly targeted to the memory location of message from message id, avoids id to the extra mapping of memory location.
The above describes a kind of storage inside structure of information storaging apparatus 200 among Fig. 2 in detail by Fig. 3, and the below returns embodiment shown in Figure 2.Information storaging apparatus 200 can send to message management equipment 400 with the information that each relevant partition holding upgrades after the message of its reception of storage, preserved by message management equipment 400.
The message of more than having introduced among the embodiment produces equipment 100 and information storaging apparatus 200, below in conjunction with the concrete processing procedure to the message under certain theme, continues to introduce message processing device 300 and message management equipment 400.Because each parts and each component information in the message management equipment 400 in the message processing device 300 are more alternately, therefore each included parts in the first summarized introduction message processing device 300 below, and each parts of comprising of message management equipment 400, then in specifying information reciprocal process, each parts that relates to are introduced in more detail again.
When message processing device 300 needs to process the message of certain Topic, at first process request by storage acquisition of information device 302 generating messages of message processing device 300, this Message Processing request comprises the theme of message processing device 300 processing messages of asking at least, then by the transmission interface 304 of message processing device 300 the Message Processing request is sent to the transmission interface 404 of message management equipment 400.
After the transmission interface 404 of message management equipment 400 receives Message Processing request from message processing device 300, this Message Processing request is sent to subregion state requestor 406, and then subregion state requestor 406 is inquired about the one or more partition holding information corresponding with the theme in this Message Processing request in message distribution status register 402.
Particularly, since store in the message distribution status register 402 message management equipment 400 that receive, from the information of the relevant partition holding of each information storaging apparatus 200, therefore distribution requestor 406 can be according to the theme of the message of appointment in the Message Processing request, and the message stores from message distribution status register 402 under this theme of inquiry is on which or which partition holding of which or which information storaging apparatus 200.For example, supposing that distribution requestor 406 inquires has 5 partition holding storing messages and processes the message under the designated key in the request, each partition holding identifies with broker_id-partition_id, be respectively: b1-p1, b1-p2, b2-p1, b2-p2, b3-p1, the partition holding information that comprises this 5 partition holdings sign that then will inquire sends to the transmission interface 304 of corresponding message processing device 300 by transmission interface 404.
The transmission interface 304 of message processing device 300 receives from after message management equipment 400, one or more partition holding information corresponding with the theme in the Message Processing request, is transmitted to message location manager 306 and processes.
Particularly, message location manager 306 is selected a partition holding information successively from one or more partition holding information, then obtains the current positional information of wanting processing messages corresponding to selected partition holding information from message management equipment 400.For example, message location manager 306 comprises selects module and message position enquiry module, at first select module from one or more partition holding information, to select successively a partition holding information, when the Selective storage partition information, can be the partition holding information of selecting at random successively from one or more partition holding information, also can be to select successively a partition holding information according to the numbering of information storaging apparatus 200 and the numbering of the partition holding on corresponding information storaging apparatus 200.Then message position enquiry module is according to the corresponding message of the partition holding Information generation position query requests of selecting module to select, comprise the selected partition holding information of module of selecting in the query requests of message position, and then by transmission interface 304 the message position query requests that message position enquiry module generates is sent to the transmission interface 404 of message management equipment 400, in order to from message management equipment 400, obtain the positional information of current message to be processed corresponding to this partition holding information.
The transmission interface 404 of message management equipment 400 is transmitted to message location manager 408 with this message position query requests after receiving message position query requests from message processing device 300.Because in message distribution status register 402, also store the positional information of the current message to be processed of relevant information treatment facility in each partition holding, after so message location manager 408 receives message position query requests, the positional information of the current message to be processed of the inquiry partition holding corresponding with the partition holding in this message position query requests from message distribution status register 402 inquires about Query Result is sent to the transmission interface 304 of corresponding message treatment facility 300 by transmission interface 404 after complete.If do not deposit the positional information of the current message to be processed of the partition holding corresponding with the partition holding in this message position query requests in the message distribution status register 402, can the initial value of the positional information of the current message to be processed of respective stored subregion be made as 0 by message location manager 408, and send to corresponding message processing device 300.In addition, also can will not exist the positional information of the current message to be processed of respective stored subregion directly to send to message processing device 300 by message location manager 408, by message processing device 300 initial value of the positional information of the current message to be processed of respective stored subregion will be made as 0.
Should be noted that, in a upper embodiment, the storage acquisition of information device 302 of message processing device 300 sends the Message Processing request to message management equipment 400 first, behind the result that acquisition message management equipment 400 returns, message location manager 306 sends message position query requests to message management equipment 400 again.In yet another embodiment, the storage acquisition of information device 302 of message processing device 300 and message location manager 306 also can send to message management equipment 400 simultaneously with Message Processing request and message position query requests.For example, suppose that message management equipment 400 receives Message Processing request and the message position query requests from message processing device 300 simultaneously, and then distribution requestor 406 is processed the message under the designated key in the request according to the Message Processing requesting query to having 5 partition holding storing messages, then message location manager 408 is according to Query Result and the message position query requests of distribution requestor 406, further inquire about the positional information of these 5 the current difference of partition holding message to be processed, the positional information with these 5 current respectively message to be processed of partition holding sends to corresponding message processing device 300 in the lump at last.
The transmission interface 304 of message processing device 300 is transmitted to message getter 308 with this positional information through message location manager 306 after the positional information of the current message to be processed that receives that return, corresponding with the partition holding in the query requests of the message position partition holding of message management equipment 400; Perhaps also can the positional information that receive directly be sent to message getter 308 by transmission interface 304.
The message getter 308 of message processing device 300 is as follows with the code sample that the message that the generates request of obtaining sends to corresponding information storaging apparatus 200:
$zkCluster＝′localhost:2181′；
$topic＝′test′；
$Consumer＝new?Kafka_Consumer($zkCluster，$topic)；
$msg＝$Consumer-＞fetchMsg()；
It will be appreciated by those skilled in the art that; the code that above pass-along message is obtained request only is an example, and any can realization sends to the code realization of information storaging apparatus 200 all in protection scope of the present invention with the message request of obtaining of message processing device 300 generations.
Need to prove that message processing device 300 message size Fetchsize to be dealt with generally are fixed values, such as 1M (million).Fetchsize can be kept at message processing device 300 this locality, in this case, when message getter 308 obtains request at generating messages, only needs to inquire about Fetchsize and get final product in the local disk of message processing device 300.It will be appreciated by those skilled in the art that, Fetchsize also can be kept in the message distribution status register 402 of message management equipment 400, in this case, can come processing messages to obtain when request from distribution storer 402 Query Informations at subregion state requestor 403, Fetchsize and the information of relevant partition holding be obtained together and sent to the message location manager 306 of message processing device 300.Certainly, also can obtain request by storage acquisition of information device 302 or the message location manager 306 generation Fetchsize of message processing device 300, the request of again Fetchsize being obtained occurs to message management equipment 400, and then message size Fetchsize to be dealt with is obtained in 402 inquiries of the message distribution status register from message management equipment 400.
In yet another embodiment, message processing device 300 also comprises message position renovator.After message handling device success processing messages, message position renovator generating messages position updating request, and via transmission interface 304 the message position updating request that generates being sent to message management equipment 400, the message position updating request comprises message size that message handling device the last time or many successes are processed and the partition holding information at this message place.
The transmission interface 404 of message management equipment 400 is transmitted to message location manager 408 after the message position updating request that receives from message processing device 300.Message location manager 408 is based on the message size in the message position updating request, in the updating message distribution storer 402, the current positional information of wanting processing messages of the partition holding corresponding with the partition holding in the message position updating request.When the renewal of the positional information offset of the current message to be processed of specific implementation, various ways can be arranged.For example, message position renovator can be at the every a piece of news of successfully handling of message handling device, just generate the message position updating request that comprises the message size of processing, as when message handling device is successfully handled message m sg1, message position renovator generates the message position updating request of the big or small len1 that comprises msg1, and the message location manager 408 of message management equipment 400 is deposited into new positional information offset+len1 among node/consumer/offset/b1-p1 in the message distribution status register 402.When message handling device is successfully handled second message m sg2, message position renovator generates the message position updating request of the big or small len2 that comprises msg2, the message location manager 408 of message management equipment 400 is added len2 at new offset again, and the content of the node/consumer/offset/b1-p1 in the updating message distribution storer 402.
If for the theme not high to security requirement, can be after message handling device be successfully handled multiple messages, again by the offset of respective stored subregion in the message distribution status register 402 of message position renovator generating messages position updating request updating message management equipment 400.For example, in the above example, also can be after message handling device be successfully handled three message m sg1, msg2, msg3, message position renovator regeneration comprises the message position updating request of three message size len1+len2+len3 and sends to message management equipment 400, the message location manager 408 of message management equipment 400 is deposited into new positional information offset+len1+len2+len3 in the message distribution status register 402 among node/consumer/offset/b1-p1, offset is upgraded, can increase some efficient like this.
Can find out by above a plurality of embodiment of the present invention, the embodiment of the invention is kept in the message management equipment by the positional information offset with the current message to be processed of relevant information treatment facility in each partition holding, need to calculate offset in this locality when having avoided the each processing messages of message processing device, can reduce the use cost of message processing device, improve the utilization ratio of its resource.
Further, because store in each information storaging apparatus 200 information of relevant partition holding at message management equipment 400, and the positional information of relevant information treatment facility 300 current message to be processed in each partition holding, therefore when message processing device 300 obtains message to be processed, the partition holding information of message on information storaging apparatus 200 under the theme that only needs to process from message management equipment 400 query messages treatment facilities 300, and message processing device 300 is in the positional information of the current message to be processed of corresponding partition holding, can send the message request of obtaining obtains corresponding message from information storaging apparatus 200 and processes, like this, even message processing device 300 breaks down, after message processing device 300 recovers normally, the position acquisition message that still can be right after success last time processing messages is processed, and has avoided re-treatment or the omission of message.
Corresponding with aforementioned one embodiment of the invention distributed message disposal system, Fig. 4 shows the process flow diagram of the information management method that the message management equipment that is suitable for according to an embodiment of the invention in the distributed message disposal system carries out.This distributed message disposal system comprises one or more such as the described message generation of Fig. 2 embodiment equipment 100, one or more such as the described information storaging apparatus 200 of Fig. 2 embodiment, one or more such as the described message processing device 300 of Fig. 2 embodiment and such as the described message management equipment 400 of Fig. 2 embodiment.This information management method is suitable for carrying out at the described message management equipment 400 of Fig. 2 embodiment.
This information management method starts from step S410, in step S410, and the Message Processing request that the receipt message treatment facility sends, this Message Processing request comprises the theme of the processing messages of asking at least.This step can be carried out by the transmission interface 404 in the aforementioned messages management equipment 400, and relevant technology realizes and can with reference to the associated description of transmission interface 404 in each embodiment in the aforementioned messages management equipment 400, repeat no more herein.
After step S410 received the Message Processing request, step S420 inquired about the partition holding information corresponding with the theme in the Message Processing request in message management equipment, and the partition holding information that inquires is sent to corresponding message processing device.Particularly, positional information owing to the current message to be processed of relevant information treatment facility in the information that usually stores relevant partition holding in each information storaging apparatus in the message management equipment and each partition holding, therefore, in this step, can be according to the theme of the message of appointment in the Message Processing request, the message stores from message management equipment under this theme of inquiry is on which or a little partition holding of which or which information storaging apparatus.For example, suppose to inquire and have the message that 5 partition holdings are stored this theme, each partition holding identifies with broker_id-partition_id, be respectively: b1-p1, b1-p2, b2-p1, b2-p2, b3-p1, the partition holding information that comprises these 5 partition holding signs that then will inquire sends to corresponding message processing device.When also storing message processing device message to be dealt with big or small in the message management equipment, also can be in the relevant partition holding information of inquiry, the size of inquiry corresponding message treatment facility message to be dealt with sends to corresponding message processing device in the lump.This step can be carried out by the subregion state requestor 406 in the aforementioned messages management equipment 400, and relevant technology realizes and can with reference to the associated description of aforementioned subregion state requestor 406 in each embodiment, repeat no more herein.
By step S420, message processing device has obtained the partition holding information of the message under its theme to be processed, therefore the subsequent message treatment facility will be according to the partition holding Information generation message position query requests of selecting, and sends to message management equipment.So, in step S430, process the message position query requests from message processing device.For example, in message management equipment, inquire about the current positional information of wanting processing messages of the partition holding corresponding with the partition holding in this message position query requests, then the positional information that inquires is sent to the corresponding message treatment facility.
When this step of specific implementation, various ways can be arranged, for example, suppose that the partition holding that message processing device obtains has 5, be respectively: b1-p1, b1-p2, b2-p1, b2-p2, b3-p1, from these 5 partition holdings, select in turn successively the request of a partition holding generating messages position enquiring, if the current positional information that will inquire about the current message to be processed of b1-p1 partition holding, then generate and comprise the message position query requests of b1-p1 partition holding to message management equipment, corresponding node/consumer/offset/b1-p1 inquires the positional information offset of the current message to be processed of b1-p1 partition holding from message management equipment in step S430, and sends to corresponding message processing device.If there is not this node in the message management equipment, then the initial value with the positional information of the current message to be processed of b1-p1 partition holding is made as 0, and sends to corresponding message processing device.In addition, also can will not exist the information of this node directly to send to message processing device, by message processing device the initial value of the current message positional information to be processed of b1-p1 partition holding is made as 0.This step can be carried out by the message location manager 408 in the aforementioned messages management equipment 400, and relevant technology realizes and can with reference to the associated description of aforementioned messages location manager 408 in each embodiment, repeat no more herein.
Therefore by step S430, message processing device has obtained the positional information of the current message to be processed of selected partition holding, can be accordingly obtains corresponding message from the respective stored subregion of information storaging apparatus, and message is processed.Because message processing device can cause the variation of the positional information of subsequent treatment message to the processing of message, so message processing device can be to message management equipment sending message position updating request.Particularly, in step S440, processing is from the message position updating request of message processing device, based on the message size in the message position updating request, in the updating message management equipment, the current positional information of wanting processing messages of the partition holding corresponding with the partition holding in the message position updating request.In the updating message management equipment during message positional information offset current to be processed of respective stored subregion, message processing device can at every a piece of news of successfully handling, just send the offset of respective stored subregion in the message position updating request updating message management equipment.For example when message processing device is successfully handled message m sg1, message management equipment is deposited into new positional information offset+len1 among the respective nodes/consumer/offset/b1-p1 of message management equipment according to the message position updating request that comprises len1 that message processing device sends; When message processing device was successfully handled second message m sg2, message management equipment was added len2 according to the message position updating request that comprises len2 that message processing device sends at new offset, and upgraded the content in the corresponding message management equipment.
If for the theme not high to security requirement, can after successfully handling multiple messages, upgrade again message processing device the offset in the corresponding message management equipment, for example, in the above example, also can be after message processing device be successfully handled three message, the message position updating request that comprises len1+len2+len3 that message management equipment sends according to message processing device, new positional information offset+len1+len2+len3 is deposited among the respective nodes/consumer/offset/b1-p1 of message management equipment, can increases some efficient like this.
Step S430 can carry out by the message location manager 408 in the aforementioned messages management equipment 400, and relevant technology realizes and can with reference to the associated description of aforementioned messages location manager 408 in each embodiment, repeat no more herein.
In above-described embodiment, owing to the information that stores relevant partition holding in each information storaging apparatus at message management equipment, and the positional information of the current message to be processed of relevant information treatment facility in each partition holding, when message processing device obtains message to be processed, the partition holding information of message on information storaging apparatus under the theme that only needs to process from message management equipment query message processing device, and message processing device is in the positional information of the current message to be processed of corresponding partition holding, can send the message request of obtaining obtains corresponding message from information storaging apparatus and processes, like this, even message processing device breaks down, after message processing device recovers normally, the position acquisition message that still can be right after success last time processing messages is processed, and has avoided re-treatment or the omission of message.
Corresponding with aforementioned one embodiment of the invention distributed message disposal system, Fig. 5 shows the process flow diagram that is suitable for according to an embodiment of the invention the message treatment method that the message processing device in the distributed message disposal system carries out.This distributed message disposal system comprises one or more such as the described message generation of Fig. 2 embodiment equipment 100, one or more such as the described information storaging apparatus 200 of Fig. 2 embodiment, one or more such as the described message processing device 300 of Fig. 2 embodiment and such as the described message management equipment 400 of Fig. 2 embodiment.This message treatment method is suitable for carrying out at the described message processing device 300 of Fig. 2 embodiment.
This message treatment method starts from step S510, and in step S510, generating messages is processed request, and the Message Processing request comprises the theme of the processing messages of asking at least.Particularly, message processing device is processed request according to message subject generating messages to be processed.This step can be carried out by the storage acquisition of information device 302 in the aforementioned messages treatment facility 300, and relevant technology realizes and can with reference to the associated description of aforementioned storage acquisition of information device 202 in each embodiment, repeat no more herein.
And then, in step S520, the Message Processing request that generates among the step S510 is sent to message management equipment, and one or more corresponding partition holding information of the theme with in the Message Processing request of receipt message management equipment transmission.After the Message Processing request was sent to message management equipment, message management equipment inquired one or more partition holding information corresponding with this theme, and sends to corresponding message processing device according to the theme of appointment in the Message Processing request.This step can be carried out by the transmission interface 304 in the aforementioned messages treatment facility 300, and relevant technology realizes and can with reference to the associated description of transmission interface 304 in each embodiment in the aforementioned messages treatment facility 300, repeat no more herein.
After this, in step S530, from one or more partition holding information that step S520 obtains, select first a partition holding information, then obtain the current positional information of wanting processing messages corresponding to selected partition holding information.Particularly, can realize in the following way: from one or more partition holding information that step S520 obtains, select successively a partition holding information (this step can be carried out by aforesaid selection module).For example, can be the partition holding information of selecting at random successively from one or more partition holding information first, also can be to select successively a partition holding information according to the numbering of information storaging apparatus and the numbering of the partition holding on corresponding information storaging apparatus; Then according to selected partition holding Information generation message position query requests and send to message management equipment, message position query requests comprises selected partition holding information, to obtain the current positional information (this step can with message position enquiry module by front carry out) of wanting processing messages corresponding with selected partition holding information from message management equipment.Step S530 can carry out by the message location manager 306 in the aforementioned messages treatment facility 300, relevant technology realizes and can with reference to the associated description of message location manager 306 in each embodiment in the aforementioned messages treatment facility 300, repeat no more herein.
Then, in step S540, the positional information generating messages of current message to be processed obtains request in the partition holding information that obtains according to step S520 and S530 and the corresponding partition holding, and the request of obtaining of this message comprises current positional information and the message size to be dealt with of wanting processing messages of the theme of the processing messages of asking, selected partition holding, this partition holding.This step can be carried out by the message getter 308 in the aforementioned messages treatment facility 300, and relevant technology realizes and can with reference to the associated description of message getter device 308 in each embodiment in the aforementioned messages treatment facility 300, repeat no more herein.
At last, in step S550, the message request of obtaining that step S540 is generated sends to the information storaging apparatus at corresponding partition holding place, and receive that the information storaging apparatus at this partition holding place returns obtain the message of request corresponding to this message.This step can be carried out by the transmission interface 304 in the aforementioned messages treatment facility 300, and relevant technology realizes and can with reference to the associated description of transmission interface 304 in each embodiment in the aforementioned messages treatment facility 300, repeat no more herein.
In addition, it is pointed out that when execution in step S520, also Message Processing request and message position query requests can be sent to message management equipment simultaneously.For example, suppose that message management equipment stores message under certain theme according to the Message Processing requesting query to having 5 partition holdings, message management equipment can also be according to the result who inquires, further inquire about these 5 the current difference of partition holding message positional informations to be processed, and answer simultaneously the request of step S520 to return to message processing device the current respectively message positional information to be processed of these 5 partition holdings.Like this, step S530 can select the positional information of current message to be processed a partition holding and the corresponding partition holding equally successively in turn from these 5 partition holdings.
Above-mentioned message treatment method can also may further comprise the steps: the message of obtaining request corresponding to message that receives is processed.For example, the information storaging apparatus corresponding message that the request of obtaining is returned according to message is resolved by message format, if parsing obtains three message and length is respectively: msg1, len1; Msg2, len2; Msg3, len3 processes successively to these three message.This step can be carried out by the message handling device in the aforementioned messages treatment facility 300, and relevant technology realizes and can with reference to the associated description of the message handling device in the aforementioned messages treatment facility 300 in each embodiment, repeat no more herein.
Above-mentioned message treatment method is further comprising the steps of: the generating messages position updating request also sends to message management equipment, and the message position updating request comprises the last time or the message size of many successes processing and the partition holding information at this message place.In this step, multiple implementation can be arranged, for example, can be at every a piece of news of successfully handling, just send comprise the message size of processing the message position updating request to message management equipment, the message positional information offset current to be processed of message management renewal of the equipment respective stored subregion.For example when successfully handling message m sg1, generation comprises the message position updating request of the big or small len1 of msg1, and sending to message management equipment, message management equipment is deposited into new positional information offset+len1 among respective nodes/consumer/offset/b1-p1; When successfully handling second message m sg2, generation comprises the message position updating request of the big or small len2 of msg2, and sending to message management equipment, message management equipment is added len2 at new offset again, and upgrades the content of respective nodes/consumer/offset/b 1-p1.
In another kind of implementation, if for the theme not high to security requirement, can be after successfully handling multiple messages, the offset of respective stored subregion in the regeneration message position updating request updating message management equipment, for example, in the above example, also three message m sg1 can handled successfully, msg2, behind the msg3, regeneration comprises the message position updating request of three message size len1+len2+len3 and sends to message management equipment, message management equipment is deposited into new positional information offset+len1+len2+len3 among respective nodes/consumer/offset/b1-p1, offset is upgraded, can increase some efficient like this.This step can be carried out by the message position renovator in the aforementioned messages treatment facility 300, and relevant technology realizes and can with reference to the associated description of message position renovator in each embodiment in the aforementioned messages treatment facility 300, repeat no more herein.
In above-described embodiment, owing to store the information of relevant partition holding in each information storaging apparatus on the message management equipment, and the positional information of the current message to be processed of relevant information treatment facility in each partition holding, therefore when message processing device obtains message to be processed, the partition holding information of message on information storaging apparatus under the theme that only needs to process from message management equipment query message processing device, and message processing device is in the positional information of the current message to be processed of corresponding partition holding, can send the message request of obtaining obtains message from information storaging apparatus and processes, like this, even message processing device breaks down, after message processing device recovers normally, the position acquisition message that still can be right after success last time processing messages is processed, and has avoided re-treatment or the omission of message.
Intrinsic not relevant with any certain computer, virtual system or miscellaneous equipment with demonstration at this algorithm that provides.Various general-purpose systems also can be with using based on the teaching at this.According to top description, it is apparent constructing the desired structure of this type systematic.In addition, the present invention is not also for any certain programmed language.Should be understood that and to utilize various programming languages to realize content of the present invention described here, and the top description that language-specific is done is in order to disclose preferred forms of the present invention.
In the instructions that provides herein, a large amount of details have been described.Yet, can understand, embodiments of the invention can be put into practice in the situation of these details not having.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, open and help to understand one or more in each inventive aspect in order to simplify this instructions, in the description to exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or the description to it sometimes in the above.Yet the method for the disclosure should be construed to the following intention of reflection: namely the present invention for required protection requires the more feature of feature clearly put down in writing than institute in each claim.Or rather, as following claims reflected, inventive aspect was to be less than all features of the disclosed single embodiment in front.Therefore, follow claims of embodiment and incorporate clearly thus this embodiment into, wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and can adaptively change and they are arranged in one or more equipment different from this embodiment the module in the equipment among the embodiment.Can be combined into a module or unit or assembly to the module among the embodiment or unit or assembly, and can be divided into a plurality of submodules or subelement or sub-component to them in addition.In such feature and/or process or unit at least some are mutually repelling, and can adopt any combination to disclosed all features in this instructions (comprising claim, summary and the accompanying drawing followed) and so all processes or the unit of disclosed any method or equipment make up.Unless in addition clearly statement, disclosed each feature can be by providing identical, being equal to or the alternative features of similar purpose replaces in this instructions (comprising claim, summary and the accompanying drawing followed).
In addition, those skilled in the art can understand, although embodiment more described herein comprise some feature rather than further feature included among other embodiment, the combination of the feature of different embodiment means and is within the scope of the present invention and forms different embodiment.For example, in the following claims, the one of any of embodiment required for protection can be used with array mode arbitrarily.
All parts embodiment of the present invention can realize with hardware, perhaps realizes with the software module of moving at one or more processor, and perhaps the combination with them realizes.It will be understood by those of skill in the art that and to use in practice microprocessor or digital signal processor (DSP) to realize according to some or all some or repertoire of parts in the equipment of the embodiment of the invention.The present invention can also be embodied as be used to part or all equipment or the device program (for example, computer program and computer program) of carrying out method as described herein.Such realization program of the present invention can be stored on the computer-readable medium, perhaps can have the form of one or more signal.Such signal can be downloaded from internet website and obtain, and perhaps provides at carrier signal, perhaps provides with any other form.
It should be noted above-described embodiment the present invention will be described rather than limit the invention, and those skilled in the art can design alternative embodiment in the situation of the scope that does not break away from claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and is not listed in element or step in the claim.Being positioned at word " " before the element or " one " does not get rid of and has a plurality of such elements.The present invention can realize by means of the hardware that includes some different elements and by means of the computing machine of suitably programming.In having enumerated the unit claim of some devices, several in these devices can be to come imbody by same hardware branch.The use of word first, second and C grade does not represent any order.Can be title with these word explanations.
The application can be applied to computer system/server, and it can be with numerous other universal or special computingasystem environment or configuration operation.The example that is suitable for well-known computing system, environment and/or the configuration used with computer system/server includes but not limited to: personal computer system, server computer system, thin client, thick client computer, hand-held or laptop devices, system, set-top box, programmable consumer electronics, NetPC Network PC, minicomputer system, large computer system based on microprocessor and comprise the distributed cloud computing technology environment of above-mentioned any system, etc.Computer system/server can be described under the general linguistic context of the computer system executable instruction (such as program module) of being carried out by computer system.Usually, program module can comprise routine, program, target program, assembly, logic, data structure etc., and they are carried out specific task or realize specific abstract data type.Computer system/server can be implemented in distributed cloud computing environment, and in the distributed cloud computing environment, task is by carrying out by the teleprocessing equipment of communication network link.In distributed cloud computing environment, program module can be positioned on the Local or Remote computing system storage medium that comprises memory device.
Claims (4)
1. message management equipment in the distributed message disposal system, this distributed message disposal system also comprises one or more information storaging apparatus and one or more message processing device, wherein every message has corresponding theme, described information storaging apparatus is that one or more partition holding set up in corresponding theme, and according to the theme of message with message stores in one of partition holding of corresponding theme, described message management equipment comprises:
The message distribution status register is configured to store at least in each information storaging apparatus the positional information of the current message to be processed of relevant information treatment facility in the information of relevant partition holding and each partition holding;
Transmission interface is configured to the Message Processing request that the receipt message treatment facility sends, and described Message Processing request comprises the theme of the processing messages of asking at least;
The subregion state requestor, be configured to the inquiry partition holding information corresponding with the theme in the described Message Processing request in described message distribution status register, and via described transmission interface the partition holding information that inquires sent to the corresponding message treatment facility; And
The message location manager, be configured to process receive by described transmission interface, from the message position query requests of message processing device, from described message distribution status register, obtain the current of the partition holding corresponding with the partition holding in this message position query requests and want the positional information of processing messages and send to the corresponding message treatment facility via described transmission interface; And be configured to process receive by described transmission interface, from the message position updating request of message processing device, based on the message size in the described message position updating request, upgrade in the described message distribution status register, the current positional information of wanting processing messages of the partition holding corresponding with the partition holding in the described message position updating request.
2. message management equipment as claimed in claim 1 also stores the size of the each message to be dealt with of message processing device in the wherein said message distribution status register;
Described subregion state requestor also is configured to the size of inquiry corresponding message treatment facility message to be dealt with in described message distribution status register, and sends to the corresponding message treatment facility by described transmission interface.
3. information management method, the message management equipment that the method is suitable in the distributed message disposal system is carried out, this distributed message disposal system also comprises one or more information storaging apparatus and one or more message processing device, wherein every message has corresponding theme, described information storaging apparatus is that one or more partition holding set up in corresponding theme, and according to the theme of message with message stores in one of partition holding of corresponding theme, described information management method comprises:
The Message Processing request that the receipt message treatment facility sends, described Message Processing request comprises the theme of the processing messages of asking at least;
Inquire about the partition holding information corresponding with the theme in the described Message Processing request, and the partition holding information that inquires sent to the corresponding message treatment facility, wherein, store at least in each information storaging apparatus the positional information of the current message to be processed of relevant information treatment facility in the information of relevant partition holding and each partition holding in the described message management equipment; And
Processing is inquired about partition holding current corresponding with the partition holding in this message position query requests and is wanted the positional information of processing messages and send to the corresponding message treatment facility from the message position query requests of message processing device; And processing is from the message position updating request of message processing device, based on the message size in the described message position updating request, upgrade the current positional information of wanting processing messages of the partition holding corresponding with the partition holding in the described message position updating request.
4. information management method as claimed in claim 3 also comprises:
The size of inquiry corresponding message treatment facility message to be dealt with also sends to the corresponding message treatment facility, also stores the size of the each message to be dealt with of message processing device in the wherein said message management equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210436385.7A CN102955717B (en) | 2012-11-05 | 2012-11-05 | Message management equipment and method in distributed message processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210436385.7A CN102955717B (en) | 2012-11-05 | 2012-11-05 | Message management equipment and method in distributed message processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102955717A true CN102955717A (en) | 2013-03-06 |
| CN102955717B CN102955717B (en) | 2015-05-20 |
Family
ID=47764546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210436385.7A Expired - Fee Related CN102955717B (en) | 2012-11-05 | 2012-11-05 | Message management equipment and method in distributed message processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102955717B (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104361065A (en) * | 2014-11-04 | 2015-02-18 | 福建亿榕信息技术有限公司 | Orderly sequence number generating method of Zookeeper-based distributed system |
| CN104754036A (en) * | 2015-03-06 | 2015-07-01 | 合一信息技术（北京）有限公司 | Message processing system and processing method based on kafka |
| CN105204776A (en) * | 2015-08-18 | 2015-12-30 | 北京京东尚科信息技术有限公司 | Data processing method and device |
| CN106170013A (en) * | 2016-06-29 | 2016-11-30 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of Kafka message uniqueness method based on Redis |
| CN106371366A (en) * | 2016-09-22 | 2017-02-01 | 南京中新赛克科技有限责任公司 | ARM architecture-based big data acquisition and analysis platform |
| CN106375462A (en) * | 2016-09-13 | 2017-02-01 | 北京百度网讯科技有限公司 | Method and device for realizing message persistence in distributed message system |
| CN107438061A (en) * | 2016-05-27 | 2017-12-05 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of kafka clients authentication |
| CN107562810A (en) * | 2017-08-10 | 2018-01-09 | 北京时代云英科技有限公司 | Video big data is classified storage method |
| CN109190025A (en) * | 2018-08-21 | 2019-01-11 | 北京京东尚科信息技术有限公司 | information monitoring method, device, system and computer readable storage medium |
| CN110162410A (en) * | 2018-02-12 | 2019-08-23 | 北京京东尚科信息技术有限公司 | A kind of message treatment method and device |
| WO2020042612A1 (en) * | 2018-08-31 | 2020-03-05 | 华为技术有限公司 | Method and device for storing and reading a message, server, and storage medium |
| CN111741041A (en) * | 2019-07-10 | 2020-10-02 | 北京京东尚科信息技术有限公司 | Message processing method and device, electronic equipment and computer readable medium |
| CN111786750A (en) * | 2020-06-30 | 2020-10-16 | 深圳前海微众银行股份有限公司 | Message query method and message receiving terminal |
| CN112380155A (en) * | 2019-09-25 | 2021-02-19 | 北京城建设计发展集团股份有限公司 | Cloud-based rail transit integrated data bus communication method and system |
| WO2021082073A1 (en) * | 2019-10-30 | 2021-05-06 | 平安科技（深圳）有限公司 | Method and system for dynamic selection of storage partitions, computer device, and storage medium |
| CN114461407A (en) * | 2022-04-13 | 2022-05-10 | 杭州涂鸦信息技术有限公司 | Data processing method, data processing device, distribution server, data processing system, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060101148A1 (en) * | 2004-11-05 | 2006-05-11 | Christian Herrmann | Method, system, and program product for monitoring message flow in a message queuing system |
| CN1988512A (en) * | 2005-12-23 | 2007-06-27 | 国际商业机器公司 | Device, method and system for supporting multimedia news sending and receiving based on application |
| CN102469033A (en) * | 2010-11-01 | 2012-05-23 | 阿里巴巴集团控股有限公司 | Message subscription system and message sending method |
| CN102668516A (en) * | 2011-12-02 | 2012-09-12 | 华为技术有限公司 | Method and apparatus for realizing message passing amid cloud information service |
-
2012
- 2012-11-05 CN CN201210436385.7A patent/CN102955717B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060101148A1 (en) * | 2004-11-05 | 2006-05-11 | Christian Herrmann | Method, system, and program product for monitoring message flow in a message queuing system |
| CN1988512A (en) * | 2005-12-23 | 2007-06-27 | 国际商业机器公司 | Device, method and system for supporting multimedia news sending and receiving based on application |
| CN102469033A (en) * | 2010-11-01 | 2012-05-23 | 阿里巴巴集团控股有限公司 | Message subscription system and message sending method |
| CN102668516A (en) * | 2011-12-02 | 2012-09-12 | 华为技术有限公司 | Method and apparatus for realizing message passing amid cloud information service |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104361065B (en) * | 2014-11-04 | 2018-07-06 | 福建亿榕信息技术有限公司 | The ordered sequence generation method of distributed system based on Zookeeper |
| CN104361065A (en) * | 2014-11-04 | 2015-02-18 | 福建亿榕信息技术有限公司 | Orderly sequence number generating method of Zookeeper-based distributed system |
| CN104754036A (en) * | 2015-03-06 | 2015-07-01 | 合一信息技术（北京）有限公司 | Message processing system and processing method based on kafka |
| CN105204776A (en) * | 2015-08-18 | 2015-12-30 | 北京京东尚科信息技术有限公司 | Data processing method and device |
| CN105204776B (en) * | 2015-08-18 | 2019-06-04 | 北京京东尚科信息技术有限公司 | Data processing method and device |
| CN107438061B (en) * | 2016-05-27 | 2020-03-03 | 北京京东尚科信息技术有限公司 | Method and device for authenticating kafka client |
| CN107438061A (en) * | 2016-05-27 | 2017-12-05 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of kafka clients authentication |
| CN106170013A (en) * | 2016-06-29 | 2016-11-30 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of Kafka message uniqueness method based on Redis |
| CN106375462A (en) * | 2016-09-13 | 2017-02-01 | 北京百度网讯科技有限公司 | Method and device for realizing message persistence in distributed message system |
| CN106371366A (en) * | 2016-09-22 | 2017-02-01 | 南京中新赛克科技有限责任公司 | ARM architecture-based big data acquisition and analysis platform |
| CN107562810A (en) * | 2017-08-10 | 2018-01-09 | 北京时代云英科技有限公司 | Video big data is classified storage method |
| CN110162410A (en) * | 2018-02-12 | 2019-08-23 | 北京京东尚科信息技术有限公司 | A kind of message treatment method and device |
| CN109190025B (en) * | 2018-08-21 | 2021-09-14 | 北京京东尚科信息技术有限公司 | Information monitoring method, device, system and computer readable storage medium |
| CN109190025A (en) * | 2018-08-21 | 2019-01-11 | 北京京东尚科信息技术有限公司 | information monitoring method, device, system and computer readable storage medium |
| WO2020042612A1 (en) * | 2018-08-31 | 2020-03-05 | 华为技术有限公司 | Method and device for storing and reading a message, server, and storage medium |
| CN111741041A (en) * | 2019-07-10 | 2020-10-02 | 北京京东尚科信息技术有限公司 | Message processing method and device, electronic equipment and computer readable medium |
| CN111741041B (en) * | 2019-07-10 | 2023-05-12 | 北京京东尚科信息技术有限公司 | Message processing method and device, electronic equipment and computer readable medium |
| CN112380155A (en) * | 2019-09-25 | 2021-02-19 | 北京城建设计发展集团股份有限公司 | Cloud-based rail transit integrated data bus communication method and system |
| WO2021082073A1 (en) * | 2019-10-30 | 2021-05-06 | 平安科技（深圳）有限公司 | Method and system for dynamic selection of storage partitions, computer device, and storage medium |
| CN111786750A (en) * | 2020-06-30 | 2020-10-16 | 深圳前海微众银行股份有限公司 | Message query method and message receiving terminal |
| CN114461407A (en) * | 2022-04-13 | 2022-05-10 | 杭州涂鸦信息技术有限公司 | Data processing method, data processing device, distribution server, data processing system, and storage medium |
| CN114461407B (en) * | 2022-04-13 | 2022-08-26 | 杭州涂鸦信息技术有限公司 | Data processing method, data processing device, distribution server, data processing system, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102955717B (en) | 2015-05-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102955717A (en) | Message management equipment and method in distributed message processing system | |
| CN102981911A (en) | Distributed message handling system and device and method thereof | |
| CN107590001B (en) | Load balancing method and device, storage medium and electronic equipment | |
| US8554832B1 (en) | Server side user interface simulation | |
| CN110462589A (en) | On-demand code execution in a local device coordinator | |
| US20170344910A1 (en) | Continuously provisioning large-scale machine learning models | |
| CN103034540A (en) | Distributed information system, device and coordinating method thereof | |
| CN107493191B (en) | Cluster node and self-scheduling container cluster system | |
| US20190132276A1 (en) | Unified event processing for data/event exchanges with existing systems | |
| CN103034541A (en) | Distributing type information system and equipment and method thereof | |
| CN114363407B (en) | Message service method and device, readable storage medium and electronic equipment | |
| CN101405723A (en) | Estimation of initial dynamic rendering control data | |
| CN112230616A (en) | Linkage control method and device and linkage middleware | |
| CN110352401A (en) | Local device coordinator with on-demand code execution capability | |
| CN102970284A (en) | User information processing method and server | |
| CN112346926B (en) | Resource state monitoring method and device and electronic equipment | |
| KR20140101370A (en) | Autonomous network streaming | |
| CN115349117B (en) | Multi-level cache grid system for multi-tenant, serverless environments | |
| CN105677536A (en) | Implementing method for task messages and task system for implementing task messages | |
| CN112751847A (en) | Interface call request processing method and device, electronic equipment and storage medium | |
| CN105162879A (en) | Method, device and system for realizing data consistency among plurality of machine rooms | |
| US20180314548A1 (en) | Work item management in content management systems | |
| CN105516333A (en) | Interactive method and system based on webpage | |
| CN103647811A (en) | A method and an apparatus for application's accessing backstage service | |
| CN108365976B (en) | Network service optimization method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150520 Termination date: 20211105 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |