WO2016197858A1 - 一种消息通知方法和设备 - Google Patents

一种消息通知方法和设备 Download PDF

Info

Publication number
WO2016197858A1
WO2016197858A1 PCT/CN2016/084595 CN2016084595W WO2016197858A1 WO 2016197858 A1 WO2016197858 A1 WO 2016197858A1 CN 2016084595 W CN2016084595 W CN 2016084595W WO 2016197858 A1 WO2016197858 A1 WO 2016197858A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
user
sent
logical node
server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/084595
Other languages
English (en)
French (fr)
Inventor
喻奎
段培乐
李闪
孙敬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Publication of WO2016197858A1 publication Critical patent/WO2016197858A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Definitions

  • the present application relates to the field of network technologies, and in particular, to a message notification method and device.
  • the distributed computing service model can be mainly divided into offline computing and real-time computing.
  • Off-line computing mainly uses batch processing to process data, which is suitable for big data processing services with high throughput, low cost, and insensitive data processing delay.
  • Real-time computing including streaming computing, quasi-real-time services, etc., refers to the use of distributed ideas and methods for real-time processing of massive "streaming" data, suitable for high-concurrency, low-latency data real-time processing services.
  • Such as open source architecture storm (a stream data processing architecture) provides streaming computing services
  • spark streaming a quasi-real-time distributed data processing architecture
  • the data processing engine part can be abstracted into a DAG (Directed Acyclic Graph) framework to process data.
  • DAG can be understood as a directed acyclic graph composed of some logical processing nodes with sequential dependencies. After the processing of the parent node is completed, the data is transmitted to the child nodes for processing until all the leaf nodes are processed, and the data input by the client is completed. It is considered to be processed by the data processing engine.
  • Hadoop a software framework capable of distributed processing of large amounts of data
  • the data warehouse tool Hive also converts user-submitted sql (Structured Query Language) into MR (Map-Reduce, map reduction)
  • the DAG is composed of DAG to process batch data.
  • Spark (a batch data processing architecture) is also designed as a DAG processing flow composed of operators of RRD (Resilient Distributed Datasets) data units.
  • the topology of Storm The topology is more typical of the DAG framework.
  • the data flow direction generally undergoes three stages of data input, data processing, and result output, wherein the DAG processing model belongs to the data processing part.
  • the data processing engine when the data unit is processed, some users want to know the status in time to proceed to the next process. This requires the data processing engine to provide a message notification framework or other means such as the client API to enable the user to know the processing in time.
  • the data granularity, latency, correctness, and fault tolerance of the message notification determine the difficulty of the data processing engine to implement the message notification mechanism.
  • the granularity and latency of data notification may become very high, and even the data granularity needs to be reduced to each data, that is, when a piece of data flows into the streaming computing engine, this After the strip data and all new data generated by this data are processed by all DAGs, the data processing engine needs to notify the user immediately, and the user makes corresponding decisions according to the processing result.
  • the purpose of the application is to provide a message notification method and device, which compares the received information with the ID in the information to be sent according to a preset rule, and sends the comparison result to one of the child logical nodes to continue comparison.
  • the logical node does not generate the information information after receiving the information
  • the last information carrying the comparison result of the ID is sent to the server, and the server determines whether the data information sent by the user is processed according to the preset rule.
  • the ID in the comparison information reaches the judgment whether the data information sent by the user is processed, and reaches the purpose of notifying the user.
  • Such processing is performed by each logical node and the server in the system, and the ID in the information is compared, and the Occupy too many resources of a device, so that the message notification mechanism can achieve higher performance.
  • a message notification method comprising:
  • the logical node adds a unique ID for each information to be sent, wherein the logical node sends each information carrying a unique ID can only be obtained by one of the following logical nodes;
  • the logical node compares the information to be sent with the ID in the received information according to a preset rule
  • the logical node sends the information to be sent carrying the comparison result to one of the child logical nodes, so that the child logical node uses the ID included in the received information and the ID and information in the information to be sent. Compare the results of the comparison;
  • the logical node When the logical node does not generate new information after processing the received information, the logical node compares the IDs in the received information, and carries the comparison result in the information sent to the server, so that the Service The device notifies the user after determining that the data information sent by the user is processed according to the preset rule.
  • the root logical node When the logical node is a root logical node, the root logical node receives and forwards data information input by the user, so that the root logical node forwards data information input by the user to the sub-logic of the root logical node.
  • the node processes the data information input by the user;
  • the ID information is not added to the data information input by the user.
  • the method further includes:
  • the logical node After the logical node receives the data information sent by the user, the logical node receives the same data information that is sent by the server and sent by the user, so that the server pairs the logic according to the preset rule. The information returned by the node carrying the comparison result is judged, and according to the judgment result, it is judged whether the data information sent by the user is processed.
  • the preset rule is that when there are even numbers of identical IDs, the even number of identical IDs are deleted;
  • a message notification method comprising:
  • the server receives information that is sent by the logical node and carries the comparison result
  • the server determines, according to a preset rule, the received information carrying the comparison result according to a preset rule, and notifies the user after determining that the data information sent by the user is processed;
  • the logical node is a logical node that does not generate new information according to the received information
  • the result of the comparison is that the logical node is generated by comparing the IDs contained in the received information.
  • the preset rule is that when there are even numbers of identical IDs, the even number of identical IDs are deleted;
  • the server determines the received comparison result according to the preset rule in a preset time, and notifies the user after determining that the data information sent by the user is processed, specifically:
  • the server determines the number of the same ID in the received comparison result within a preset time. If the number of the same ID is an even number, the server deletes the even number of the same ID, and the ID does not exist in the server. When the data information sent by the user is processed, and the user is notified;
  • the server retains one of the odd number of identical IDs, and continues to receive the information carrying the comparison result, and when there is an ID in the server within a predetermined time, it indicates The data sent by the user is not processed, and the data information sent by the user is notified to be completed when the ID does not exist in the server before the end of the preset time.
  • the method further includes:
  • the server After the user sends the data information to the logical node, the server sends the same data information that is sent by the user to the logical node;
  • the server receives information returned by the logical node and carrying the comparison result
  • the server determines the comparison result in the information according to a preset rule in a preset time, so that the server determines whether the data information sent by the user is processed according to the determination result.
  • a logical node device comprising:
  • Adding a module configured to add a unique ID for each information to be sent, wherein the logical node sends each information carrying a unique ID can only be obtained by one of the following logical nodes;
  • a comparison module configured to compare the information to be sent and the ID in the received information according to a preset rule
  • a first sending module configured to send, to the one of the child logical nodes, the information to be sent that carries the comparison result, so that the child logical node uses the ID included in the received information and the information to be sent The ID is compared with the comparison result;
  • a second sending module when the logical node does not generate new information after processing the received information, the logical node compares the IDs in the received information, and carries the comparison result in the information sent to the server So that the server notifies the user after the processing of the data information sent by the user is determined according to the preset rule.
  • the root logical node When the logical node is a root logical node, the root logical node receives and forwards data information input by the user, so that the root logical node forwards data information input by the user to the sub-logic of the root logical node.
  • the node processes the data information input by the user;
  • the ID information is not added to the data information input by the user.
  • the device further includes:
  • the receiving module is configured to receive the same data information that is sent by the server and sent by the user, so that the server is configured according to the preset rule.
  • the information returned by the logical node carrying the comparison result is judged, and according to the judgment result, it is judged whether the data information sent by the user is processed.
  • the preset rule is that when there are even numbers of identical IDs, the even number of identical IDs are deleted;
  • a server device comprising:
  • a first receiving module configured to receive, by the logical node, the received information and the information to be sent that carries the comparison result
  • a first determining module configured to determine, according to a preset rule, the received information carrying the comparison result according to a preset rule
  • a notification module configured to notify the user after determining that the data information sent by the user is processed
  • the logical node is a logical node that does not generate new information according to the received information
  • the result of the comparison is that the logical node is generated by comparing the IDs contained in the received information.
  • the preset rule is that when there are even numbers of identical IDs, the even number of identical IDs are deleted;
  • the first determining module is specifically configured to:
  • the server Determining, in a preset time, the number of the same IDs in the received comparison result. If the number of the same IDs is an even number, the server deletes the even number of identical IDs, and when there is no ID in the server, it indicates The data information sent by the user is processed;
  • the server retains one of the odd number of identical IDs, and continues to receive the information carrying the comparison result, and when there is an ID in the server within a predetermined time, it indicates The data sent by the user is not processed, and the data information sent by the user is notified to be completed when the ID does not exist in the server before the end of the preset time.
  • the device further includes:
  • a sending module after the user sends the data information to the logical node, to send the same data information that is sent by the user to the logical node;
  • a second receiving module configured to receive information that is returned by the logical node and that carries a comparison result
  • the second determining module is configured to determine, according to the preset rule, the comparison result in the information according to the preset rule, so that the server determines, according to the determination result, whether the data information sent by the user is processed.
  • the application compares the received information with the ID in the information to be sent according to a preset rule, and sends the information carrying the comparison result to one of the next group of logical nodes to continue comparison, when the logical node receives
  • the comparison result of the last ID is carried in the information sent to the server, and the server determines whether the data information sent by the user is processed according to the preset rule.
  • This application achieves the ID in the comparison information. Determine whether the data information sent by the user is processed and reach the purpose of notifying the user. Such processing is performed by each logical node and the server in the system, and the ID in the information is compared, and the device is not occupied too much. Resources, so that the message notification mechanism can achieve higher performance.
  • FIG. 1 is a schematic diagram of a message notification method in an embodiment of the present application
  • FIG. 3 is a second schematic diagram of a message notification method in an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a stream framework of a distributed streaming computing system according to an embodiment of the present application
  • FIG. 5 is a schematic diagram of a record data processing topology based on a stream system processing according to an embodiment of the present application
  • FIG. 6 is a schematic diagram of a working principle based on a Batch mechanism in an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a logical node in an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a server in an embodiment of the present application.
  • the logical node exists in the server, and the server includes a root logical node and a multi-level logical node under the root logical node, and the root logical node and other multi-level logical nodes may be collectively referred to as a logical node, wherein if a logic A node has a subordinate logical node, and the subordinate logical node is called a sub logical node of the logical node. As shown in FIG.
  • the server includes: a root logical node PN (Processer Node) and a multi-level logical node PN1-PN6, and PN1 and PN2 are called subordinate logical nodes of the root logical node PN, that is, sub-logic of the PN.
  • PN Root logical node
  • PN1 and PN2 are called subordinate logical nodes of the root logical node PN, that is, sub-logic of the PN.
  • the root logical node PN is called the upper logical node of the logical nodes PN1 and PN2, PN1 and PN2 can be called the first logical node, and the lower logical nodes of PN1 are PN3, PN4, PN5, PN3, PN4, PN5 are called
  • the child logical node of PN1, PN6 is the lower logical node of PN2, that is, the child logical node of PN2 is PN6, PN2 is the superior logical node of PN6, and PN3, PN4, PN5 and PN6 can be called secondary logical nodes.
  • the data information sent by the user is forwarded by the server to the root logical node, and the server records the data information sent by the user.
  • a message notification method based on a distributed computing system as shown in FIG. 1, the method includes the following steps:
  • Step 101 the logical node adds a unique ID (Identity, identity number) for each information to be sent. code).
  • the information that the logical node sends each carrying a unique ID can only be obtained by one of the child logical nodes of the logical node.
  • the IDs added by PN1 in the three information to be sent are r3, r4, and r5, respectively, wherein each information in PN1 can only be sent to the child logical nodes PN3, PN4 of PN1,
  • r3 information is added to PN3, r4 information is added to PN4, and r5 information is added to PN5.
  • the logical node processes the received information according to a preset logical calculation manner to generate information to be sent.
  • the logical calculation method used by the logical node to process the received information is preset.
  • the logical calculations used by different logical nodes may be different.
  • Step 102 The logical node compares the information to be sent with the ID in the received information according to a preset rule.
  • the root logical node When the logical node is a root logical node, the root logical node receives and forwards data information input by the user, so that the root logical node forwards data information input by the user to the child logical node of the root logical node. Processing data information input by the user;
  • the data information input by the user is generated according to a preset fixed format edit.
  • the ID information is not added to the user input data, and after the root logical node receives the data information input by the user, when the data information is sent to the child logical node of the root logical node, An ID is added to the data information.
  • the preset rule is that when there are even numbers of identical IDs, the even number of identical IDs are deleted;
  • the preset rule may also be an exclusive OR algorithm, and may also be other methods or algorithms, but any technology that can achieve the present application can be achieved.
  • the method or algorithm of the effect belongs to the protection scope of the present application.
  • the PN is a root logical node, and the PN receives data information input by the user, and the ID does not exist in the data information input by the user, but when the PN is sent to the sub-logical nodes PN1 and PN2, An ID is added to the data information. If the ID of the information transmitted to PN1 is r1 and the ID of the information transmitted to PN2 is r2, the comparison result of the IDs of the information received and transmitted by the root logical node is r1 and r2.
  • the PN1 logically calculates the information with the ID r1 according to a preset logical calculation manner, and adds an ID to the information obtained through the logical calculation, wherein the logic is added.
  • the calculated information is the information to be sent. If the ID in the information to be sent is r3, r4, r5, the pass After comparison, the information received by PN1 and the ID in the information to be transmitted are not the same, and the comparison result obtained by PN1 is r1, r3, r4, r5.
  • Step 103 The logical node sends the information to be sent carrying the comparison result to one of the child logical nodes, so that the child logical node uses the ID included in the received information and the information to be sent. The ID is compared to the comparison result.
  • the information received by the logical node includes the ID assigned by the superior logical node for the information and the result of comparing the ID included in the information received by the superior logical node with the ID in the information to be sent, and the comparison result is an ID. set.
  • the PN sends the information carrying r1 to PN1, and also sends the comparison result (r1, r2) to PN1, and the ID in the information to be transmitted by PN1 is r3, r4, r5, so PN1 There is an even number of r1, so it is deleted, and the comparison result is r2, r3, r4, r5.
  • a message carrying the ID and the comparison result can be sent to one of the logical nodes PN3, PN4, PN5.
  • Step 104 When the logical node does not generate new information after processing the received information, the logical node compares the IDs in the received information, and carries the comparison result in the information sent to the server, to And causing the server to notify the user after determining that the data information sent by the user is processed according to the preset rule.
  • PN3 does not generate new information after processing the received information
  • the ID of the information received by PN3 is r3, and the received comparison result is r2, r3, r4, r5, and an even number.
  • PN4 does not generate new information after processing the received information, and PN4 does not receive the comparison result of PN1 transmission. Only the received information has an ID in the PN4, and therefore, the PN4 will receive the information.
  • the ID (r4) is sent to the server.
  • the method further includes:
  • the logical node After the logical node receives the data information sent by the user, the logical node receives the same data information that is sent by the server and sent by the user, so that the server pairs the logic according to the preset rule. The information returned by the node carrying the comparison result is judged, and according to the judgment result, it is judged whether the data information sent by the user is processed.
  • the server cannot notify the user in time, and the server receives the data information sent by the user after the PN receives the data information according to a preset mechanism, and the PN receives the One or more data information sent by the server and sent by the user, and performing the same operation as the data information sent by the user, after the server receives all the comparison results, according to a preset rule Whether the same data information sent by the root logical node and the user is processed is completed. If the processing is completed, the data information sent by the user is also completed, but the user is not notified in time due to the data loss of the server. The user is notified that the data information sent by the user is processed.
  • the logical node directly sends the ID of the information to be sent and the ID of the received information to the server, and the server compares all the received IDs according to a preset rule to determine the data information sent by the user. Whether the processing is completed.
  • a message notification method as shown in FIG. 3, the method includes the following steps:
  • Step 301 The server receives information that is sent by the logical node and carries the comparison result.
  • the logical node is a logical node that does not generate new information according to the received information.
  • the result of the comparison is that the logical node is generated by comparing the IDs contained in the received information.
  • Step 302 The server determines, according to a preset rule, the received information carrying the comparison result according to a preset rule, and notifies the user after determining that the data information sent by the user is processed.
  • the preset rule is that when there are even numbers of identical IDs, the even number of identical IDs are deleted;
  • the logical nodes PN3, PN4, PN5, and PN6 are logical nodes that no longer generate information, indicating that the data information sent by the user is processed, then the logical nodes PN3, PN4, PN5 and The PN 6 compares the received ID in the received information with the received comparison result according to a preset rule, and transmits the comparison result to the server.
  • the server determines, according to a preset rule, the received information carrying the comparison result according to the preset rule, and notifies the user after determining that the data information sent by the user is processed, specifically:
  • the server determines, in a preset time, the number of the same IDs in the received information carrying the comparison result, and if the number of the same IDs is an even number, the server deletes the even number of the same IDs, when the server If the ID does not exist, it indicates that the data information sent by the user is processed, and the user is notified;
  • the server retains one of the odd number of identical IDs, and continues to receive the information carrying the comparison result, and when there is an ID in the server within a predetermined time, it indicates The data sent by the user is not processed, and the data information sent by the user is notified to be completed when the ID does not exist in the server before the end of the preset time.
  • the logical nodes PN3, PN4, PN5, and PN6 are logical nodes that no longer generate information
  • the PN3 compares the ID in the received information with the received comparison result: r2, r4, R5, PN4 is: r4, PN5 is: r5, point PN6 is: r2, and the logical nodes PN3, PN4, PN5, PN6 will be compared.
  • the server sends the server. At this time, the IDs appearing in the server are all evenly deleted, indicating that the data information sent by the user is processed, so the server can notify the user according to the comparison result.
  • the method further includes:
  • the server receives information returned by the logical node and carrying the comparison result
  • the server determines the comparison result in the information according to a preset rule in a preset time, so that the server determines whether the data information sent by the user is processed according to the determination result.
  • the application compares the received information with the ID in the information to be sent according to a preset rule, and sends the information carrying the comparison result to one of the next group of logical nodes to continue comparison, when the logical node receives
  • the comparison result of the last ID is carried in the information sent to the server, and the server determines whether the data information sent by the user is processed according to the preset rule.
  • This application achieves the ID in the comparison information. Determine whether the data information sent by the user is processed and reach the purpose of notifying the user. Such processing is performed by each logical node and the server in the system, and the ID in the information is compared, and the device is not occupied too much. Resources, so that the message notification mechanism can achieve higher performance.
  • a distributed streaming computing system framework includes three functional components: a control node controller, a data source shard (which can be understood as a root logical node), and a data processing logical topology topology, and A message synchronization node Acker Node for message synchronization and a batch data synchronization node GlobalBatch Syner, wherein Topology is a user-built DAG logic flow chart for processing data, which is composed of one or more logical nodes PN having sequential dependencies. Each PN can have multiple physical running instances. The data flowing into the topology flows to each PN for processing according to the path of the DAG.
  • the shard is used to receive data information sent by the user. It first writes the data to the distributed file system.
  • the Controller is configured to accept a create topology request sent by the user, and then perform necessary conversion processing to be scheduled to run on a specific one or more machines, wherein the distributed streaming computing system framework exists in the server.
  • the specific data information record processing process is as follows: the user inputs a record from the client to the server, first selects a shard node in the server to land, and then forwards it to the N topologies corresponding to the shard. Process it.
  • the process of the new record can be understood as generating a data information tree record tree triggered by the source record.
  • the shape of the tree generated by each source record is not necessarily the same, but their DAG shape is definitely a subtree of topology DAG, and the root node is the same.
  • the new record generated by the source record triggers a forest.
  • the record forest is not growing, that is, no new record is generated, the source record can be considered to be finished by the stream engine, and the user can be notified of the end state of the processing.
  • different record forests may be generated due to different data information carried by different records.
  • each newly generated record can be assigned one. Randomly generated unique ID.
  • a record is created by a PN, the record will be received by another PN.
  • the same ID corresponding to the two PNs will be 0 when performing an exclusive OR operation, indicating that the record is The PN was successfully processed.
  • all records correspond to a pair of identical IDs, so the result obtained by the XOR operation is definitely 0.
  • FIG. 5 is a flowchart of a record processing based on a stream system according to an embodiment of the present application, wherein FIG. 5 shows a record forest generated by two records processed by a stream engine, and the first image shows the topology.
  • the shape of the forest, the latter two figures are the record forest shape generated by two records: r1 and r2.
  • Each record forest has only leaf nodes (logical nodes that do not generate new information) to communicate with the Acker Node to reduce network overhead. All leaf nodes of a record forest will send the results of the local XOR partial ID calculations to The same Acker Node, since these records are derived from the same source record, they will have a message notification field with the same value as the source record called Mesg ID, and get the ID number to be sent to the acker Node through the Mesg ID. After the acker node obtains the partial exclusive value AckVal of a Mesg ID, it will XOR the value with the existing ackval corresponding to the Mesg ID saved in the current memory. If the value is 0, the Mesg ID is The corresponding source record is processed by the stream engine. The Mesg ID can be used to notify the user of the status result. If the value is not 0, it indicates that it has not been processed yet, and can continue to wait.
  • the user submits two requests to create a topology: top1 and top2, and specifies to process the data on shard1.
  • the Controller After the user request is sent to the Controller, the Controller generates the necessary floor plans and then schedules the two topologies in the computing cluster in order.
  • the PN of each topology can be performed by thread granularity or process granularity, depending on the implementation mechanism of the resource scheduling system used by the cluster.
  • the Controller randomly generates an integer value (ID) for each topology, and the first PN of each topology also carries the integer.
  • the system When top2 is created, the system randomly assigns the integer t2 to the acker Node.
  • the ackval t1 ⁇ t2 ( ⁇ indicates the XOR operator), and the ackval value is the result
  • PN3 receives r3, no new record is generated, so it belongs to a leaf node of record tree.
  • the ackval 0 of the mesgID1 corresponding to the source record(r1) on the acker node, the source r1 is considered to have been The completion is completed, and the record forest corresponding to r1 does not grow anymore. At this time, mesgID1 can be notified to the user, and the user can make the next decision after obtaining the result.
  • the above embodiment tracks the structure of the record tree based on the XOR principle.
  • Each record only needs to add 8 bytes to save the ackval value (the stream system uses 8 bytes by default, and can of course be extended to 16 bytes to reduce The same record probability that different records in the same record forest are randomly generated, and the ackval is carried to the leaf node to send the final local ackval to the Acker Node for final calculation, only need to consume the necessary network resources.
  • Figure 6 shows a flow diagram of an auxiliary fault-tolerant mechanism to handle exceptions. Based on this mechanism, the Acker Node does not need to persist any state, and the record forest leaf node does not need to re-transmit the ackval to the Acker Node. At the same time, it can ensure that the mesgID corresponding to all source records will not be lost in any case, and the user can be notified in time.
  • This fault-tolerant mechanism solves these problems by using the concept of Batch.
  • One batch will contain several records, all shards.
  • the incremental Batch ID will be distributed periodically, and all the incoming records belong to a Batch until the next Batch ID arrives.
  • the Batch ID will flow into all the PNs of all theology in a sequential order as a special record.
  • the Topology PN can be subdivided into a root node agent PN and a non-root node inner PN, the agentPN is the root node of the topology DAG, and the innerPN is the other node of the topology DAG. All the topologies connected to the same shard are needed for optimization purposes.
  • the agentPNs are all placed in the same process.
  • Each data source instance shard instance corresponds to an agent PN process.
  • the agent PNs of top1 and top2 are placed in the same process, and the agent PN of each topology is located.
  • the process is an agent node.
  • the Mesg ID of the record is immediately uploaded. Notifying the user, or when it receives a Global Batch ID End message, all the records belonging to the Batch ID and less than the Batch ID that have not been notified to the user are sent to notify the user that the records have been processed.
  • InnerPN refers to all other nodes of topology, not a certain PN node.
  • Blobal Batch Syner is a single point (BBS), which is used to globally synchronize the end of the Batch ID message.
  • BBS single point
  • a Global Batch ID End message is generated for the Batch ID and sent to all Acker Node and Mesg Notifier for all Agent PNs.
  • the Global Batch ID End indicates that the stream engine has processed all the records belonging to the Batch ID and less than the Batch ID.
  • the Mesg Notifier can naturally notify the user of these records.
  • the Acker Node receives the Global Batch ID End, it will clear all the records belonging to the Batch ID and less than the Batch ID. It is not necessary to maintain their status, because these records have been processed, and MesgNotifier will notify the user of these records. .
  • an instance of shard will send batch1 in order, source record(r1, r2, r3), batch2, source record(r4, r5) to its connected agent node process; process
  • the agent PN and Mesg Notifier of top1 and top2 will receive all of this data, Mesg Notifier will save the data into memory until notification to the user, the other agent PN will process the data and then generate the new record and The original batch ID is sent to the child node.
  • each topology will process the record tree corresponding to r1, r2, and r3 at a certain time.
  • the ackval in the record tree leaf node of r1, r2, and r3 will be used.
  • all the leaf nodes of topology will send the batch1End message to the BBS single point.
  • the Mesg Notifier first receives the r2 and r3 sent by the Acker Node, and informs the user that the two records have been processed, as shown by the number 5 in FIG. 6, and the batch1 has the r1. The user has not been notified yet.
  • Mesg Notifier receives the Global Batch1End message sent by the BBS, it notifies the user of r1. Until now, all the records in batch1 have been sent to the user without losing any record.
  • Acker Node receives the message of Global Batch1End, it will clear the ackval state of r1, no need to maintain it, and save memory resources.
  • the present application compares the received information with the ID in the information to be sent according to a preset rule by the logical node. Comparing, and sending the comparison result to one of the next group of logical nodes to continue comparison, when the logical node no longer generates information information after receiving the information, the comparison result of the last ID is sent to the server, and the server according to the preset rule Determining whether the data information sent by the user is processed is completed.
  • the present application determines whether the data information sent by the user is processed by comparing the ID in the information, and achieves the purpose of notifying the user.
  • the processing manner is shared by each logical node and the server in the system. Completed, and compares the ID in the information, does not occupy too many resources of a certain device, so that the message notification mechanism can achieve relatively high performance, and does not need to know the state of information processing on each logical node to save a lot of Internet resources.
  • the present application also proposes a logical node device. As shown in FIG. 7, the device includes:
  • An adding module 71 configured to add a unique ID for each information sent, where the logical node sends each information carrying a unique ID can only be obtained by one of the following logical nodes;
  • the comparing module 72 is configured to compare the information to be sent and the ID in the received information according to a preset rule
  • the first sending module 73 is configured to send the comparison result to one of the next group of logical nodes, so that the logical node that receives the comparison result compares the received information with the ID in the information to be sent with the comparison result. ;
  • a second sending module 74 when the logical node does not generate new information after processing the received information, sending a result of comparing the IDs in the received information to the server, so that the server Determining, according to the preset rule, whether the data information sent by the user is processed.
  • the information received by the root logical node is data information sent by the user;
  • the ID information is not added to the data information sent by the user.
  • the device further includes:
  • the receiving module is configured to receive the same data information that is sent by the server and sent by the user, so that the server is configured according to the preset rule.
  • the comparison result returned by the logical node is judged, and according to the judgment result, it is judged whether the data information sent by the user is processed.
  • the preset rule is that when there are even numbers of identical IDs, the even number of identical IDs are deleted;
  • the present application also proposes a server device, as shown in FIG. 8, the device includes:
  • the first receiving module 81 is configured to receive, by the logical node, the received information and the ID in the information to be sent. The result of the comparison;
  • the first determining module 82 is configured to determine, according to a preset rule, the received comparison result in a preset time
  • the notification module 83 is configured to notify the user after determining that the processing of the data information sent by the user is completed;
  • the logical node is a logical node that does not generate new information according to the received information.
  • the preset rule is that when there are even numbers of identical IDs, the even number of identical IDs are deleted;
  • the first determining module 81 is specifically configured to:
  • the server Determining, in a preset time, the number of the same IDs in the received comparison result. If the number of the same IDs is an even number, the server deletes the even number of identical IDs, and when there is no ID in the server, it indicates The data information sent by the user is processed;
  • the server retains one of the odd number of identical IDs, and continues to receive the comparison result, indicating that the user sends the ID when there is an ID in the server within a predetermined time
  • the data is not processed, and the user is notified that the outgoing information processing sent by the user is completed until the ID is not present in the server before the end of the preset time.
  • the device further includes:
  • a sending module after the user sends the data information to the logical node, to send the same data information that is sent by the user to the logical node;
  • a second receiving module configured to receive a comparison result returned by the logical node
  • the second determining module is configured to determine the comparison result according to the preset rule in a preset time, so that the server determines, according to the determination result, whether the data information sent by the user is processed.
  • the application compares the received information with the ID in the information to be sent according to a preset rule, and sends the information carrying the comparison result to one of the next group of logical nodes to continue comparison, when the logical node receives
  • the comparison result of the last ID is carried in the information sent to the server, and the server determines whether the data information sent by the user is processed according to the preset rule.
  • This application achieves the ID in the comparison information. Determine whether the data information sent by the user is processed and reach the purpose of notifying the user. Such processing is performed by each logical node and the server in the system, and the ID in the information is compared, and the device is not occupied too much. Resources, so that the message notification mechanism can achieve higher performance.
  • the portion may be embodied in the form of a software product stored in a storage medium, including instructions for causing a terminal device (which may be a cell phone, a personal computer, a server, or a network device, etc.) to execute the application.
  • a terminal device which may be a cell phone, a personal computer, a server, or a network device, etc.
  • modules in the apparatus in the embodiments may be distributed in the apparatus of the embodiment according to the description of the embodiments, or the corresponding changes may be located in one or more apparatuses different from the embodiment.
  • the modules of the foregoing embodiments may be integrated into one or may be deployed separately; may be combined into one module, or may be further split into multiple sub-modules.
  • the serial numbers of the embodiments of the present application are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)
  • Computer And Data Communications (AREA)

Abstract

本申请公开了一种消息通知方法和设备，该方法包括：逻辑节点为将要发送的每个信息添加一个唯一ID；逻辑节点根据预设规则用将要发送的信息和接收的信息中的ID进行比较；逻辑节点将携带有比较结果的所述将要发送的信息发送给子逻辑节点中的一个；当逻辑节点对接收的信息进行处理后没有产生新的信息时，所述逻辑节点将接收的信息中的ID进行比较，并将比较结果携带在发送给所述服务器的信息中，以使所述服务器在根据所述预设规则判断出用户发送的数据信息处理完成后通知用户。本申请通过上述方法能在处理完用户的数据信息后成功的通知用户，同时还能减少网络和内存的开销。

Description

一种消息通知方法和设备
本申请要求2015年6月12日递交的申请号为201510325555.8、发明名称为“一种消息通知方法和设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及网络技术领域，特别是涉及一种消息通知方法和设备。
背景技术
目前分布式计算服务模型主要可分为离线计算和实时计算两种。离线计算主要采用批处理的方式处理数据，适用于高吞吐，低成本，数据处理延迟不敏感的大数据处理业务。实时计算包括流式计算，准实时服务等，是指利用分布式的思想和方法，对海量“流”式数据进行实时处理，适用于高并发，低延时的数据实时处理业务。如开源架构storm(一种流数据处理架构)提供的流式计算服务，spark streaming(一种准实时分布式数据处理架构)提供的准实时服务等。不管是哪种计算模型，在分布式的环境下，数据处理引擎部分基本可抽象成DAG(Directed Acyclic Graph，有向无环图)的框架来处理数据。DAG可以理解为有顺序依赖关系的一些逻辑处理节点组成的有向无环图，数据在父节点处理完成之后才会传送给子节点进行处理，直到所有的叶子节点处理完成，客户端输入的数据才算被数据处理引擎处理完成了。如hadoop(一种能够对大量数据进行分布式处理的软件框架)的数据仓库工具Hive也是将用户提交的sql(Structured Query Language，结构化查询语言)转换成由MR(Map-Reduce，映射归约)组成的DAG来处理批量数据，spark(一种批量数据处理架构)也设计成是基于RDD(Resilient Distributed Datasets，弹性分布式数据集)数据单元的算子组成的DAG处理流程，storm的拓扑结构topology更加是典型的DAG框架。
上述系统中，可以认为数据流向大体上都会经历数据输入，数据处理，结果输出三个阶段，其中DAG处理模型就属于数据处理部分。在有些业务场景下，当数据单元被处理完成后，有些用户希望及时知道这个状态以进行下一步流程，这就需要数据处理引擎提供消息通知框架或者其他方式如客户端API让用户能够及时知晓处理结果，而消息通知的数据粒度，延时性，正确性以及容错性决定了数据处理引擎实现消息通知机制的难度。对于基于DAG架构的离线计算，用户可通过调用客户端的API(客户端的操作系 统留给应用程序的一个调用接口)获取到输入的一批数据是否处理完成的情况，达到通知用户的目的，大部分离线计算业务对于消息通知的数据粒度和及时性要求很低，因此一般不需要严格意义上及时的采用推送PUSH模式的消息通知机制。而对于很多基于DAG架构的流式计算业务，对数据通知的粒度和延时性可能会变得很高，甚至数据粒度需要缩小到每条数据，即当一条数据流入流式计算引擎后，这条数据以及由这条数据触发而产生的所有新数据被所有的DAG处理完成后，数据处理引擎需要立马通知用户，用户根据处理结果及时作出相应决策。
在实现本申请的过程中，申请人发现现有技术至少存在以下问题：
基于目前的分布式计算系统还没有出现比较成熟高效的消息通知机制，并且由于某些分布式计算系统框架本身的缺陷使消息通知机制无法实现预期的效果。
发明内容
本申请的目的在于提供一种消息通知方法和设备，通过逻辑节点根据预设规则对接收的信息和将要发送的信息中的ID进行比较，并将比较结果发送给子逻辑节点中的一个继续比较，当逻辑节点在接收到信息后不再产生信息的信息时将最后的携带有ID的比较结果的信息发送给服务器，由服务器根据预设规则判断用户发送的数据信息是否处理完成，本申请通过比较信息中的ID达到判断用户发送的数据信息是否处理完成，并达到通知用户的目的，这样的处理方式由系统中的各个逻辑节点和服务器共同完成，并且比较的是信息中的ID，不会占用某一设备过多的资源，从而使消息通知机制能够达到比较高的性能。
本申请的技术方案如下：
一种消息通知方法，所述方法包括：
所述逻辑节点为将要发送的每个信息添加一个唯一ID，其中，所述逻辑节点发送每个携带有唯一ID的信息只能被下组逻辑节点中的一个获取；
所述逻辑节点根据预设规则用将要发送的信息和接收的信息中的ID进行比较；
所述逻辑节点将携带有比较结果的所述将要发送的信息发送给子逻辑节点中的一个，以使所述子逻辑节点用接收的信息中包含的ID和将要发送的信息中的ID与所述比较结果进行比较；
当逻辑节点对接收的信息进行处理后没有产生新的信息时，所述逻辑节点将接收的信息中的ID进行比较，并将比较结果携带在发送给所述服务器的信息中，以使所述服务 器在根据所述预设规则判断出用户发送的数据信息处理完成后通知用户。
当所述逻辑节点为根逻辑节点时，所述根逻辑节点接收并转发所述用户输入的数据信息，以使所述根逻辑节点将用户输入的数据信息转发给所述根逻辑节点的子逻辑节点来对用户输入的数据信息进行处理；
其中，所述用户输入的数据信息中没有添加ID。
所述方法还包括：
在所述逻辑节点接收到用户发送的数据信息后，所述逻辑节点接收所述服务器发送的与所述用户发送的相同的数据信息，以使所述服务器根据所述预设规则对所述逻辑节点返回的携带有比较结果的信息进行判断，并根据判断结果判断所述用户发送的数据信息是否处理完成。
所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
一种消息通知方法，所述方法包括：
所述服务器接收逻辑节点发送的携带有比较结果的信息；
所述服务器在预设时间内根据预设规则对接收的携带有比较结果的信息进行判断，在判断出用户发送的数据信息处理完成后通知所述用户；
其中，所述逻辑节点为不会在根据接收的信息产生新信息的逻辑节点；
所述比较结果是所述逻辑节点用接受的信息中包含的ID进行比较生成的。
所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
所述服务器在预设时间内根据预设规则对接收的比较结果进行判断，在判断出用户发送的数据信息处理完成后通知所述用户，具体为：
所述服务器在预设时间内判断接收的比较结果中相同ID的个数，如果相同ID的个数为偶数个，所述服务器删除所述偶数个相同的ID，当所述服务器中不存在ID时则表示所述用户发送的数据信息处理完成，并通知所述用户；
如果相同ID的个数为奇数个，所述服务器保留所述奇数个相同ID中的一个，并继续接收携带有比较结果的信息，当在预定时间内所述服务器中还存在ID时则表示所述用户发送的数据没有处理完成，直到所述预设时间结束前所述服务器中不存在ID时通知所述用户其发送的数据信息处理完成。
所述方法还包括：
在用户向逻辑节点发送数据信息后，所述服务器向所述逻辑节点发送与所述用户发送的相同的数据信息；
所述服务器接收逻辑节点返回的携带有比较结果的信息；
所述服务器在预设时间内根据预设规则对所述信息中的比较结果进行判断，以使所述服务器根据判断结果判断所述用户发送的数据信息是否处理完成。
一种逻辑节点设备，所述设备包括：
添加模块，用于为将要发送的每个信息添加一个唯一ID，其中，所述逻辑节点发送每个携带有唯一ID的信息只能被下组逻辑节点中的一个获取；
比较模块，用于根据预设规则用将要发送的信息和接收的信息中的ID进行比较；
第一发送模块，用于将携带有比较结果的所述将要发送的信息发送给子逻辑节点中的一个，以使所述子逻辑节点用接收的信息中包含的ID和将要发送的信息中的ID与所述比较结果进行比较；
第二发送模块，当逻辑节点对接收的信息进行处理后没有产生新的信息时，所述逻辑节点将接收的信息中的ID进行比较，并将比较结果携带在发送给所述服务器的信息中，以使所述服务器在根据所述预设规则判断出用户发送的数据信息处理完成后通知用户。
当所述逻辑节点为根逻辑节点时，所述根逻辑节点接收并转发所述用户输入的数据信息，以使所述根逻辑节点将用户输入的数据信息转发给所述根逻辑节点的子逻辑节点来对用户输入的数据信息进行处理；
其中，所述用户输入的数据信息中没有添加ID。
所述设备还包括：
接收模块，在所述逻辑节点接收到用户发送的数据信息后，用于接收所述服务器发送的与所述用户发送的相同的数据信息，以使所述服务器根据所述预设规则对所述逻辑节点返回的携带有比较结果的信息进行判断，并根据判断结果判断所述用户发送的数据信息是否处理完成。
所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
一种服务器设备，所述设备包括：
第一接收模块，用于接收逻辑节点发送的用接收的信息和将要发送的携带有比较结果的信息；
第一判断模块，用于在预设时间内根据预设规则对接收的携带有比较结果的信息进行判断；
通知模块，用于在判断出用户发送的数据信息处理完成后通知所述用户；
其中，所述逻辑节点为不会在根据接收的信息产生新信息的逻辑节点；
所述比较结果是所述逻辑节点用接受的信息中包含的ID进行比较生成的。
所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
所述第一判断模块，具体用于：
在预设时间内判断接收的比较结果中相同ID的个数，如果相同ID的个数为偶数个，所述服务器删除所述偶数个相同的ID，当所述服务器中不存在ID时则表示所述用户发送的数据信息处理完成；
如果相同ID的个数为奇数个，所述服务器保留所述奇数个相同ID中的一个，并继续接收携带有比较结果的信息，当在预定时间内所述服务器中还存在ID时则表示所述用户发送的数据没有处理完成，直到所述预设时间结束前所述服务器中不存在ID时通知所述用户其发送的数据信息处理完成。
所述设备还包括：
发送模块，在用户向逻辑节点发送数据信息后，用于向所述逻辑节点发送与所述用户发送的相同的数据信息；
第二接收模块，用于接收所述逻辑节点返回的携带有比较结果的信息；
第二判断模块，用于在预设时间内根据预设规则对所述信息中的比较结果进行判断，以使所述服务器根据判断结果判断所述用户发送的数据信息是否处理完成。
本申请通过逻辑节点根据预设规则对接收的信息和将要发送的信息中的ID进行比较，并将携带有比较结果的信息发送给下组逻辑节点中的一个继续比较，当逻辑节点在接收到信息后不再产生新的信息时将最后的ID的比较结果携带在发送给服务器的信息中，由服务器根据预设规则判断用户发送的数据信息是否处理完成，本申请通过比较信息中的ID达到判断用户发送的数据信息是否处理完成，并达到通知用户的目的，这样的处理方式由系统中的各个逻辑节点和服务器共同完成，并且比较的是信息中的ID，不会占用某一设备过多的资源，从而使消息通知机制能够达到比较高的性能。
附图说明
为了更清楚地说明本申请或现有技术中的技术方案，下面将对本申请或现有技术描述中所需要使用的附图作简单的介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例中的一种消息通知方法示意图之一；
图2为本申请实施例中的一种信息处理流程示意图；
图3为本申请实施例中的一种消息通知方法示意图之二；
图4为本申请实施例中的一种分布式流式计算系统stream框架结构示意图；
图5为本申请实施例中的一种基于stream系统处理的record数据处理拓扑示意图；
图6为本申请实施例中的一种基于Batch机制的工作原理示意图；
图7为本申请实施例中的一种逻辑节点的结构示意图；
图8为本申请实施例中的一种服务器的结构示意图。
具体实施方式
下面将结合本申请中的附图，对本申请中的技术方案进行清楚、完整的描述，显然，所描述的实施例是本申请的一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员获得的其他实施例，都属于本申请保护的范围。
逻辑节点存在于服务器中，所述服务器中包括一个根逻辑节点和所述根逻辑节点下的多级逻辑节点，根逻辑节点和其他多级逻辑节点都可以统称为逻辑节点，其中，如果一个逻辑节点有下级逻辑节点，那么下级逻辑节点称为该逻辑节点的子逻辑节点，。如图2所示，服务器中包括：一个根逻辑节点PN(Processer Node，逻辑节点)和多级逻辑节点PN1-PN6，PN1和PN2称为根逻辑节点PN的下级逻辑节点，即PN的子逻辑节点，同时，根逻辑节点PN称为逻辑节点PN1和PN2的上级逻辑节点，PN1和PN2可以称为一级逻辑节点，PN1的下级逻辑节点为PN3、PN4、PN5，PN3、PN4、PN5称为PN1的子逻辑节点，PN6为PN2的下级逻辑节点，即PN2的子逻辑节点为PN6，PN2为PN6的上级逻辑节点，PN3、PN4、PN5和PN6可以称为二级逻辑节点。
用户发送的数据信息由所述服务器转发到根逻辑节点上，同时所述服务器记录所述用户发送的数据信息。
一种基于分布式计算系统的消息通知方法，如图1所示，所述方法包括以下步骤：
步骤101，逻辑节点为将要发送的每个信息添加一个唯一ID(Identity，身份标识号 码)。
其中，所述逻辑节点发送每个携带有唯一ID的信息只能被所述逻辑节点的子逻辑节点中的一个获取。
具体的，如图2所示，PN1在其要发送3个信息中添加的ID分别为r3、r4、r5，其中，PN1中的每个信息只能发送到PN1的子逻辑节点PN3、PN4、PN5中的一个中，例如：添加有r3信息被发送到PN3中，添加有r4信息被发送到PN4中，添加有r5信息被发送到PN5中。
其中，所述逻辑节点根据预先设定的逻辑计算方式对接收的信息进行处理生成将要发送的信息，具体的，逻辑节点对接受的信息进行处理时使用的逻辑计算方式是预先设定好的，并且，不同的逻辑节点使用的逻辑计算方式也可能是不同的。
步骤102，所述逻辑节点根据预设规则用将要发送的信息和接收的信息中的ID进行比较。
当所述逻辑节点为根逻辑节点时，所述根逻辑节点接收并转发用户输入的数据信息，以使所述根逻辑节点将用户输入的数据信息转发给所述根逻辑节点的子逻辑节点来对用户输入的数据信息进行处理；
所述用户输入的数据信息是根据预先设定的固定格式编辑生成的。
其中，所述用户输入的数据信息中没有添加ID，在所述根逻辑节点接收到所述用户输入的数据信息后，在将所述数据信息发送给所述根逻辑节点的子逻辑节点时在所述数据信息中添加ID。
所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个，当然所述预设规则也可以为异或算法，还可以为其他方法或算法，但是凡是能够达到本申请技术效果的方法或算法均属于本申请的保护范围。
具体的，如图2所示，PN为根逻辑节点，所述PN接收用户输入的数据信息，用户输入的数据信息中不存在ID，但是所述PN发送给子逻辑节点PN1和PN2时，在所述数据信息中添加ID，如果发送给PN1的信息的ID为r1发送给PN2的信息的ID为r2，那么，根逻辑节点接受和发送的信息的ID的比较结果是r1、r2。
如果PN1接收的信息中包含的ID为r1，所述PN1根据预先设定的逻辑计算方式对ID为r1的信息进行逻辑计算，并在通过逻辑计算后得到的信息中添加ID，其中，通过逻辑计算得到的信息为将要发送的信息，如果将要发送的信息中的ID为r3、r4、r5，通 过比较，PN1接收的信息和将要发送的信息中的ID没有相同的，此时PN1得到的比较结果是r1、r3、r4、r5。
步骤103，所述逻辑节点将携带有比较结果的所述将要发送的信息发送给子逻辑节点中的一个，以使所述子逻辑节点用接收的信息中包含的ID和将要发送的信息中的ID与所述比较结果进行比较。
其中，逻辑节点接收的信息中包含有上级逻辑节点为所述信息分配的ID和上级逻辑节点接收的信息中包含的ID与将要发送的信息中的ID进行比较的结果，所述比较结果一个ID集合。
具体的，如图2所示，PN将携带有r1的信息发送给PN1，同时也将比较结果(r1，r2)发送给PN1，PN1将要发送信息中的ID为r3、r4、r5，因此PN1中有偶数个r1，因此将其删除，比较结果为r2、r3、r4、r5，在PN1可以将携带有ID的一个信息和比较结果发送给逻辑节点PN3、PN4、PN5中的一个。
步骤104，当逻辑节点对接收的信息进行处理后没有产生新的信息时，所述逻辑节点将接收的信息中的ID进行比较，并将比较结果携带在发送给所述服务器的信息中，以使所述服务器在根据所述预设规则判断出用户发送的数据信息处理完成后通知用户。
具体的，如图2所示，PN3在对接收到的信息进行处理后没有产生新的信息，PN3接收的信息的ID为r3，接收的比较结果是r2、r3、r4、r5，有偶数个r3，因此将r3删除，比较结果为r2、r4、r5，PN3将此结果携带在发送服务器的信息中；
PN4在对接收到的信息进行处理后没有产生新的信息，且PN4没有接收到PN1发送的比较结果，所述PN4中只有接收到得信息中有ID，因此，所述PN4将接收的信息中的ID(r4)发送给服务器。
所述方法还包括：
在所述逻辑节点接收到用户发送的数据信息后，所述逻辑节点接收所述服务器发送的与所述用户发送的相同的数据信息，以使所述服务器根据所述预设规则对所述逻辑节点返回的携带有比较结果的信息进行判断，并根据判断结果判断所述用户发送的数据信息是否处理完成。
具体的，为了保证服务器由于特殊原因导致的数据丢失而不能及时的通知用户，所述服务器根据预先设定的机制，所述服务器在判断出PN接收到用户发送的数据信息后，PN接收所述服务器发送的一个或多个与所述用户发送的相同的数据信息，并进行与所述用户发送的数据信息相同的操作，在所述服务器接收到所有比较结果后根据预设规则判 断根逻辑节点接收的与所述用户发送的相同的数据信息是否处理完成，如果处理完成那么表示之前用户发送的数据信息同样完成了，只是由于服务器的数据丢失导致没有及时通知用户，此时服务器需要通知用户其发送的数据信息处理完成。
当然，所述逻辑节点将将要发送的信息的ID和接收的信息的ID直接发送给所述服务器，由所述服务器根据预设规则对接收所有的ID进行比较来判断所述用户发送的数据信息是否处理完成。
一种消息通知方法，如图3所示，所述方法包括以下步骤：
步骤301，所述服务器接收逻辑节点发送的携带有比较结果的信息。
其中，所述逻辑节点为不会在根据接收的信息产生新信息的逻辑节点。
所述比较结果是所述逻辑节点用接受的信息中包含的ID进行比较生成的。
步骤302，所述服务器在预设时间内根据预设规则对接收的携带有比较结果的信息进行判断，在判断出用户发送的数据信息处理完成后通知所述用户。
所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
具体的，如图2所示，逻辑节点PN3、PN4、PN5和PN6为不再产生信息的逻辑节点，则表示所述用户发送的数据信息处理完成，那么所述逻辑节点PN3、PN4、PN5和PN6根据预设规则将接收的信息中ID和接收的比较结果进行比较后，将比较结果发送给所述服务器。
所述服务器在预设时间内根据预设规则对接收的携带有比较结果的信息进行判断，在判断出用户发送的数据信息处理完成后通知所述用户，具体为：
所述服务器在预设时间内判断接收的携带有比较结果的信息中相同ID的个数，如果相同ID的个数为偶数个，所述服务器删除所述偶数个相同的ID，当所述服务器中不存在ID时则表示所述用户发送的数据信息处理完成，并通知所述用户；
如果相同ID的个数为奇数个，所述服务器保留所述奇数个相同ID中的一个，并继续接收携带有比较结果的信息，当在预定时间内所述服务器中还存在ID时则表示所述用户发送的数据没有处理完成，直到所述预设时间结束前所述服务器中不存在ID时通知所述用户其发送的数据信息处理完成。
具体的，如图2所示，逻辑节点PN3、PN4、PN5、PN6为不再产生信息的逻辑节点，并且PN3将接受的信息中的ID和接收的比较结果进行比较后为：r2、r4、r5，PN4为：r4，PN5为：r5，点PN6为：r2，逻辑节点PN3、PN4、PN5、PN6将比较后的结果发 送给服务器，此时所述服务器中出现的ID都是偶数个能够删除，表示用户发送的数据信息处理完成，因此所述服务器可以根据比较结果来通知所述用户。
同时，由于用户发送一个数据信息的处理周期是有限制的，在一个处理周期内如果没有完成处理，则表示处理超时或者系统出现问题。
在用户向逻辑节点发送数据信息后，所述方法还包括：
所述服务器向逻辑节点发送与所述用户发送的相同的数据信息；
所述服务器接收逻辑节点返回的携带有比较结果的信息；
所述服务器在预设时间内根据预设规则对所述信息中的比较结果进行判断，以使所述服务器根据判断结果判断所述用户发送的数据信息是否处理完成。
本申请通过逻辑节点根据预设规则对接收的信息和将要发送的信息中的ID进行比较，并将携带有比较结果的信息发送给下组逻辑节点中的一个继续比较，当逻辑节点在接收到信息后不再产生新的信息时将最后的ID的比较结果携带在发送给服务器的信息中，由服务器根据预设规则判断用户发送的数据信息是否处理完成，本申请通过比较信息中的ID达到判断用户发送的数据信息是否处理完成，并达到通知用户的目的，这样的处理方式由系统中的各个逻辑节点和服务器共同完成，并且比较的是信息中的ID，不会占用某一设备过多的资源，从而使消息通知机制能够达到比较高的性能。
为了进一步阐述本申请的技术思想，现结合具体的应用场景，其中预设规则为异或运算，对本申请的技术方案进行说明。
具体的，如图4所示的一种分布式流式计算系统框架，其中包括三个功能组件：控制节点Controller，数据源Shard(可以理解为根逻辑节点)和数据处理逻辑拓扑图Topology，和用于消息同步的消息同步节点Acker Node和批量数据同步节点GlobalBatch Syner，其中，Topology是一个用户构建的处理数据的DAG逻辑流程图，它由具有顺序依赖关系的一个或多个逻辑节点PN构成，每个PN可以有多个物理运行实例，流入topology的数据会按照DAG的路径顺序流到各个PN上进行处理，Shard用于接收用户发送的数据信息，它先把数据写到分布式文件系统中，保证数据正常落地，然后再发送给后面连接的N个topology进行处理。Controller用来接受用户发送的创建topology请求，然后进行必要的转换处理后会调度到具体的一台或多台机器上运行，其中，所述分布式流式计算系统框架存在与服务器中。
具体的数据信息record处理过程如下：用户从客户端向服务器中输入一条record，先会选择服务器中某个shard节点落地，然后被转发到该shard对应的N个topology上面 进行处理。每个topology的PN在处理完接受到的record后可能会产生N个(N>＝0)新的record继续发往它的子PN节点进行处理，就这样随着topology DAG的计算路径不断产生的新record的过程可以理解为由源头record触发生成了一个数据信息树record tree，每条源record产生的tree形状不一定相同，但是它们的DAG形状肯定是topology DAG的子树，而且根节点相同。如果shard后面连接的topology有多个，那么源头record触发生成的新record就会组成一个森林。当这个record森林不在生长，即不再产生新的record时，可认为这条源record被stream引擎处理结束了，可以通知用户处理结束的状态。同时，由于不同record携带的数据信息不同，可能会生成不同的record森林。
当一个数据信息被处理完成时，其中的信息总是成对出现的，即一个逻辑节点发送一个信息，该信息就会被另一个逻辑节点接收，因此，可以每个新生成的record都会指定一个随机生成的唯一ID，当一个record被一个PN创建，那么该record就会被另一个PN接收，这两个PN对应的同一个ID，在进行异或运算时就会为0，说明这个record被PN成功处理了，进一步的，当一个record森林完成的时候，所有record都对应一对相同的ID，因此通过异或运算得到的结果肯定为0。
如图5所示为本申请实施例中一种基于stream系统处理的record的流程图，其中，图5显示了先后两条record经过stream引擎处理后生成的record森林，第一张图显示了topology森林形状，后面两张图分别是两条record：r1和r2生成的record森林形状。
每个record森林只有叶子节点(不会产生新信息的逻辑节点)会跟Acker Node进行通信以减少网络开销，一个record森林所有的叶子节点都会把本地先通过异或部分ID计算出来的结果发往同一个Acker Node，由于这些record都是由同一源record派生出来的，因此他们都会有一个和源record相同值的消息通知字段称之为Mesg ID，通过Mesg ID得到要发送给acker Node的ID编号，acker Node获取到某个Mesg ID的部分异或值AckVal后，会把这个值和当前内存中保存的此Mesg ID对应的已有的ackval进行异或运算，如果值为0，说明该Mesg ID对应的源record经过stream引擎处理完成了，可以将此Mesg ID通知用户状态结果，如果值不为0，说明还没有处理完，可继续等待。
具体的，用户提交两个创建topology的请求：top1和top2，并且指定处理shard1上的数据。用户请求发送到了Controller后，Controller生成必要的平面图之后会把两个topology在计算集群中按顺序调度起来。每个topology的PN可以线程粒度或者进程粒度来执行，具体什么方式取决于集群使用的资源调度系统的实现机制。Controller会针对每个topology随机生成一个整数值(ID)，每个topology第一个PN也会携带这个整数作 为从shard上接收的源record的初始ackval，例如：当创建top1时，随机分配整数t1,acker Node上所有源record的初始ackval＝t1，当创建top2时，系统随机分配整数t2，acker Node上的ackval＝t1^t2(^表示异或运算符)，ackval值是异或运算后的结果。
在top1和top2调度起来之后，并且r1的消息通知字段值为mesgId1，落地之后把r1转发给top1的第一个PN1和top2第一个PN2，PN2接收到r1后给它赋初值ackval＝t1，由于top2只有一个逻辑节点PN2，因此PN2就是叶子节点，即top2的record tree中r1就是叶子节点，因此把PN2上r1的ackval＝t1发送给acker Node，由于acker Node上所有record的初始ackval＝t1^t2，将它和发送过来的t1进行异或运算得到ackval＝t1^t2^t1＝t2，结果不为0，因此acker Node继续等待mesgID1的其他ackval发送过来；PN1接收到r1后给它赋初值ackval＝t2，然后r1被PN2处理完之后会生成三条record(r3,r4,r5)，系统会分别为其随机分配初始ackval(t3，t4，5)，PN2会默认选择最后一个生成的record(r5)携带其他新生成record(r3，r4)的ackval(t3，t4)和接受到的record(r1)的ackval(t2)的异或运算值作为它的ackval，因此r5的ackval＝t2^t3^t4，r3的ackval＝t3,r4的ackval＝t4,r5不需要异或自己的ackval(t5)，因为它发送到下一个PN后，这个t5最终会被异或为0，因此不需要这个多余的异或运算。r3，r4，r5接着被分别发送给PN3，PN4，PN5。
所有没有直接或间接依赖关系的PN，都是并行运行的，顺序不分先后，所以PN3，PN4，PN5会并行处理发送过来的record。PN3接收到r3,没有再产生新的record，因此属于record tree的一个叶子节点，PN3会把r3携带的ackval＝t3发送给acker Node后，此时mesgId1的ackval＝t2^t3；PN4接收r4，产生了新的record(r6)，此时r6就是最后一个新生成的record，按照上述步骤可得r6需携带的ackval＝t4，并将r6发往下一个PN6；PN5接收到r5之后会产生两个新的record(r7，r8)，系统为第一个新的record r7分配新的初始ackval＝t7，最后r7ackval＝t7，按照上述步骤r7的ackval(t7)和r5的ackval(t2^t3^t4)会发送给r8携带下去，因此r8ackval＝t2^t3^t4^t7，r7和r8分别发往PN7和PN8处理。
PN6接收到r6之后，没有再产生新的节点，属于record tree的一个叶子节点，因此会把r6携带的ackval＝t4发送给acker Node后，此时mesgId1的ackval＝t2^t3^t4；PN7接收到r7之后，没有再产生新的节点，会把r7携带的ackval＝t7发送给acker Node后，此时mesgId1的ackval＝t2^t3^t4^t7；PN8接收到r8之后，没有再产生新的节点，会把r8携带的ackval＝t2^t3^t4^t7发送给acker Node后，此时mesgId1的ackval＝t2^t3^t4^t7^t2^t3^t4^t7＝0。
最后acker Node上源record(r1)对应的mesgID1的ackval＝0，则认为源r1已经被处 理完成了，并且r1对应的record森林确实不再生长了，此时可以将mesgID1通知给用户，用户获取结果后可进行下一步的决策。
上述实施例基于异或原理跟踪record tree的结构，每个record只需要增加8个字节来保存ackval值即可(stream系统默认使用8个字节，当然也可以扩展为16个字节以减少同一个record森林中不同record随机产生的ID却相同的概率)，而且通过record携带ackval到叶子节点才发送最后的本地ackval到Acker Node进行最终的计算，只需要消耗一次必要的网络资源即可。
如图6所示为一个辅助的容错机制来处理异常情况的流程示意图，基于这套机制，Acker Node不需要持久化任何状态，record森林叶子节点也不需要进行网络重发ackval给Acker Node，但同时又能保证所有源record对应的mesgID在任何情况下都不会丢失可及时通知用户，这套容错机制借助了Batch的概念来解决这些问题，其中，一个Batch会包含若干条record，所有的shard会被定期分发递增的Batch ID，并且流入的所有record都属于一个Batch，直到下一个Batch ID到来；Batch ID会作为一种特殊的record按顺序流入到所有的topology的所有PN，每个PN会同步所有的父PN的Batch ID是否都已经到来，如果是，则认为该Batch结束，这个PN不会再有属于这个Batch ID的record到来了，然后生成Batch IDEnd的消息发往所有的子PN，每个PN都是同样的处理流程，直到叶子PN。
其中，Topology的PN可细分为根节点agent PN和非根节点inner PN，agentPN即为topology DAG的根节点，innerPN为topology DAG的其他节点，为了优化需要，连接在同一个shard上面的所有topology的agentPN都放在同一个进程中，每个数据源实例shard instance对应一个agent PN进程，如图6所示，top1和top2的agent PN放在同一个进程内，每个topology的agent PN所在的进程为agent Node，该agentNode进程中还存在一个Mesg Notifier，它是消息通知框架中连接用户的组件，当它被Acker Node告知某条record已经被处理完成时，会立马把这条record的Mesg ID通知给用户，或者当它接受到一个Global Batch ID End的消息时，会把属于该Batch ID和小于该Batch ID的所有还没有通知给用户的record全部发出去通知用户这些record已处理完成。InnerPN代指topology的其他所有节点，并不是指某个PN节点。Blobal Batch Syner是一个单点(BBS)，用来全局同步Batch ID结束的消息，当topology的叶子PN节点生成一个Batch IDEnd的消息时会发送给BBS，当BBS接受到来自所有的topology的所有叶子PN的Batch IDEnd消息时，会针对该Batch ID生成一个Global Batch ID End的消息，并发送给所有的 Acker Node和所有的Agent PN的Mesg Notifier。Global Batch ID End表示stream引擎已经处理完了属于这个Batch ID和小于这个Batch ID的所有record，Mesg Notifier自然可以将这些record通知给用户。同时Acker Node接收到Global Batch ID End后会将属于这个Batch ID和小于这个Batch ID的所有record清除，不需要维护他们的状态了，因为这些record已经处理完了，并且MesgNotifier会把这些record通知给用户。
如图6中的标号1所示，shard的某个instance将按顺序依次发送batch1，源record(r1，r2，r3)，batch2，源record(r4,r5)给它连接的agent Node进程；进程中的top1和top2的agent PN和Mesg Notifier都会接收到所有的这些数据，Mesg Notifier会保存这些数据到内存中，直到通知给用户，其他agent PN会将这些数据处理完后把生成的新record和原始batch ID发送给子节点。
如图6中的标号2所示，在某一时刻每个topology会依次处理完r1，r2，r3对应的record tree，在这过程中会把r1，r2，r3的record tree叶子节点中的ackval依次发送给Acker Node做异或运算。当batch1中所有record都处理完成了，topology的所有叶子节点会把batch1End消息发送给BBS单点。
如图6中的标号3所示，top1的r1，r2，r3的record tree的所有ackval都发送给了Acker Node，top2的r2，r3的record tree的所有ackval都发送给了Acker Node，而top2的r1的record tree的所有叶子节点的ackval在发往Acker Node的过程中由于网络等原因可能丢失了一部分，因此到最后Acker Node通过异或运算后，只能得到r2和r3的ackval为0，r1的值不为0，因此AckerNode只会把r2，r3告知Mesg Notifier通知给用户，至于r1则需要另外的机制即Batch机制来保证通知给用户。
当BBS接受完了所有topology的所有叶子PN的Batch1End结束的消息后，则生成Global Batch1End的消息，到此时为止，batch1所有的record已经被处理完毕，因此这个Global Batch1End会被发往mesg Notifier和Acker Node。
如图6中的标号4所示，Mesg Notifier先接收到Acker Node发送过来的r2，r3，通知用户这两条record已处理完成，如图6中的标号5所示，此时batch1还有r1还没有通知用户，当Mesg Notifier接受到BBS发送过来的Global Batch1End的消息后，再把r1通知给用户，到现在为止，batch1中所有的record已经发送给了用户，没有丢失任何一条record。当Acker Node接受到Global Batch1End的消息后，则会清理r1对应ackval状态，不需要再维护它了，节省内存资源。
本申请通过逻辑节点根据预设规则对接收的信息和将要发送的信息中的ID进行比 较，并将比较结果发送给下组逻辑节点中的一个继续比较，当逻辑节点在接收到信息后不再产生信息的信息时将最后的ID的比较结果发送给服务器，由服务器根据预设规则判断用户发送的数据信息是否处理完成，本申请通过比较信息中的ID达到判断用户发送的数据信息是否处理完成，并达到通知用户的目的，这样的处理方式由系统中的各个逻辑节点和服务器共同完成，并且比较的是信息中的ID，不会占用某一设备过多的资源，从而使消息通知机制能够达到比较高的性能，并且不用获知每个逻辑节点上信息处理的状态节省了大量的网络资源。
基于与上述方法同样的申请构思，本申请还提出了一种逻辑节点设备，如图7所述，该设备包括：
添加模块71，用于为发送的每个信息添加一个唯一ID，其中，所述逻辑节点发送每个携带有唯一ID的信息只能被下组逻辑节点中的一个获取；
比较模块72，用于根据预设规则用将要发送的信息和接收的信息中的ID进行比较；
第一发送模块73，用于将比较结果发送给下组逻辑节点中的一个，以使接收所述比较结果的逻辑节点用接收的信息和将要发送的信息中的ID与所述比较结果进行比较；
第二发送模块74，当所述逻辑节点对接收的信息进行处理后没有产生新的信息时，用于将接收的信息中的ID进行比较后的结果发送给所述服务器，以使所述服务器根据所述预设规则判断用户发送的数据信息是否处理完成。
当所述逻辑节点为根逻辑节点时，所述根逻辑节点接收的信息为所述用户发送的数据信息；
其中，所述用户发送的数据信息中没有添加ID。
所述设备还包括：
接收模块，在所述逻辑节点接收到用户发送的数据信息后，用于接收所述服务器发送的与所述用户发送的相同的数据信息，以使所述服务器根据所述预设规则对所述逻辑节点返回的比较结果进行判断，并根据判断结果判断所述用户发送的数据信息是否处理完成。
所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
奇于与上述方法同样的申请构思，本申请还提出了一种服务器设备，如图8所述，该设备包括：
第一接收模块81，用于接收逻辑节点发送的用接收的信息和将要发送的信息中的ID 进行比较后的结果；
第一判断模块82，用于在预设时间内根据预设规则对接收的比较结果进行判断；
通知模块83，用于在判断出用户发送的数据信息处理完成后通知所述用户；
其中，所述逻辑节点为不会在根据接收的信息产生新信息的逻辑节点。
所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
所述第一判断模块81，具体用于：
在预设时间内判断接收的比较结果中相同ID的个数，如果相同ID的个数为偶数个，所述服务器删除所述偶数个相同的ID，当所述服务器中不存在ID时则表示所述用户发送的数据信息处理完成；
如果相同ID的个数为奇数个，所述服务器保留所述奇数个相同ID中的一个，并继续接收比较结果，当在预定时间内所述服务器中还存在ID时则表示所述用户发送的数据没有处理完成，直到所述预设时间结束前所述服务器中不存在ID时通知所述用户其发送的出局信息处理完成。
所述设备还包括：
发送模块，在用户向逻辑节点发送数据信息后，用于向所述逻辑节点发送与所述用户发送的相同的数据信息；
第二接收模块，用于接收所述逻辑节点返回的比较结果；
第二判断模块，用于在预设时间内根据预设规则对所述比较结果进行判断，以使所述服务器根据判断结果判断所述用户发送的数据信息是否处理完成。
本申请通过逻辑节点根据预设规则对接收的信息和将要发送的信息中的ID进行比较，并将携带有比较结果的信息发送给下组逻辑节点中的一个继续比较，当逻辑节点在接收到信息后不再产生新的信息时将最后的ID的比较结果携带在发送给服务器的信息中，由服务器根据预设规则判断用户发送的数据信息是否处理完成，本申请通过比较信息中的ID达到判断用户发送的数据信息是否处理完成，并达到通知用户的目的，这样的处理方式由系统中的各个逻辑节点和服务器共同完成，并且比较的是信息中的ID，不会占用某一设备过多的资源，从而使消息通知机制能够达到比较高的性能。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献 的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台终端设备(可以是手机，个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
以上所述仅是本申请的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视本申请的保护范围。
本领域技术人员可以理解实施例中的装置中的模块可以按照实施例描述进行分布于实施例的装置中，也可以进行相应变化位于不同于本实施例的一个或多个装置中。上述实施例的模块可以集成于一体，也可以分离部署；可以合并为一个模块，也可以进一步拆分成多个子模块。上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
以上公开的仅为本申请的几个具体实施例，但是，本申请并非局限于此，任何本领域的技术人员能思之的变化都应落入本申请的保护范围。

Claims (16)

  1. 一种消息通知方法，其特征在于，所述方法包括：
    所述逻辑节点为将要发送的每个信息添加一个唯一ID，其中，所述逻辑节点发送每个携带有唯一ID的信息只能被下组逻辑节点中的一个获取；
    所述逻辑节点根据预设规则用将要发送的信息和接收的信息中的ID进行比较；
    所述逻辑节点将携带有比较结果的所述将要发送的信息发送给子逻辑节点中的一个，以使所述子逻辑节点用接收的信息中包含的ID和将要发送的信息中的ID与所述比较结果进行比较；
    当逻辑节点对接收的信息进行处理后没有产生新的信息时，所述逻辑节点将接收的信息中的ID进行比较，并将比较结果携带在发送给所述服务器的信息中，以使所述服务器在根据所述预设规则判断出用户发送的数据信息处理完成后通知用户。
  2. 如权利要求1所述方法，其特征在于，当所述逻辑节点为根逻辑节点时，所述根逻辑节点接收并转发所述用户输入的数据信息，以使所述根逻辑节点将用户输入的数据信息转发给所述根逻辑节点的子逻辑节点来对用户输入的数据信息进行处理；
    其中，所述用户输入的数据信息中没有添加ID。
  3. 如权利要求1所述方法，其特征在于，所述方法还包括：
    在所述逻辑节点接收到用户发送的数据信息后，所述逻辑节点接收所述服务器发送的与所述用户发送的相同的数据信息，以使所述服务器根据所述预设规则对所述逻辑节点返回的携带有比较结果的信息进行判断，并根据判断结果判断所述用户发送的数据信息是否处理完成。
  4. 如权利要求1所述方法，其特征在于，所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
    当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
  5. 一种消息通知方法，其特征在于，所述方法包括：
    所述服务器接收逻辑节点发送的携带有比较结果的信息；
    所述服务器在预设时间内根据预设规则对接收的携带有比较结果的信息进行判断，在判断出用户发送的数据信息处理完成后通知所述用户；
    其中，所述逻辑节点为不会在根据接收的信息产生新信息的逻辑节点；
    所述比较结果是所述逻辑节点用接受的信息中包含的ID进行比较生成的。
  6. 如权利要求5所述方法，其特征在于，所述预设规则为当有偶数个相同的ID时， 则删除所述偶数个相同的ID；
    当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
  7. 如权利要求6所述方法，其特征在于，所述服务器在预设时间内根据预设规则对接收的比较结果进行判断，在判断出用户发送的数据信息处理完成后通知所述用户，具体为：
    所述服务器在预设时间内判断接收的比较结果中相同ID的个数，如果相同ID的个数为偶数个，所述服务器删除所述偶数个相同的ID，当所述服务器中不存在ID时则表示所述用户发送的数据信息处理完成，并通知所述用户；
    如果相同ID的个数为奇数个，所述服务器保留所述奇数个相同ID中的一个，并继续接收携带有比较结果的信息，当在预定时间内所述服务器中还存在ID时则表示所述用户发送的数据没有处理完成，直到所述预设时间结束前所述服务器中不存在ID时通知所述用户其发送的数据信息处理完成。
  8. 如权利要求5所述方法，其特征在于，所述方法还包括：
    在用户向逻辑节点发送数据信息后，所述服务器向所述逻辑节点发送与所述用户发送的相同的数据信息；
    所述服务器接收逻辑节点返回的携带有比较结果的信息；
    所述服务器在预设时间内根据预设规则对所述信息中的比较结果进行判断，以使所述服务器根据判断结果判断所述用户发送的数据信息是否处理完成。
  9. 一种逻辑节点设备，其特征在于，所述设备包括：
    添加模块，用于为将要发送的每个信息添加一个唯一ID，其中，所述逻辑节点发送每个携带有唯一ID的信息只能被下组逻辑节点中的一个获取；
    比较模块，用于根据预设规则用将要发送的信息和接收的信息中的ID进行比较；
    第一发送模块，用于将携带有比较结果的所述将要发送的信息发送给子逻辑节点中的一个，以使所述子逻辑节点用接收的信息中包含的ID和将要发送的信息中的ID与所述比较结果进行比较；
    第二发送模块，当逻辑节点对接收的信息进行处理后没有产生新的信息时，所述逻辑节点将接收的信息中的ID进行比较，并将比较结果携带在发送给所述服务器的信息中，以使所述服务器在根据所述预设规则判断出用户发送的数据信息处理完成后通知用户。
  10. 如权利要求9所述设备，其特征在于，当所述逻辑节点为根逻辑节点时，所述 根逻辑节点接收并转发所述用户输入的数据信息，以使所述根逻辑节点将用户输入的数据信息转发给所述根逻辑节点的子逻辑节点来对用户输入的数据信息进行处理；
    其中，所述用户输入的数据信息中没有添加ID。
  11. 如权利要求9所述设备，其特征在于，所述设备还包括：
    接收模块，在所述逻辑节点接收到用户发送的数据信息后，用于接收所述服务器发送的与所述用户发送的相同的数据信息，以使所述服务器根据所述预设规则对所述逻辑节点返回的携带有比较结果的信息进行判断，并根据判断结果判断所述用户发送的数据信息是否处理完成。
  12. 如权利要求9所述设备，其特征在于，所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
    当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
  13. 一种服务器设备，其特征在于，所述设备包括：
    第一接收模块，用于接收逻辑节点发送的用接收的信息和将要发送的携带有比较结果的信息；
    第一判断模块，用于在预设时间内根据预设规则对接收的携带有比较结果的信息进行判断；
    通知模块，用于在判断出用户发送的数据信息处理完成后通知所述用户；
    其中，所述逻辑节点为不会在根据接收的信息产生新信息的逻辑节点；
    所述比较结果是所述逻辑节点用接受的信息中包含的ID进行比较生成的。
  14. 如权利要求13所述设备，其特征在于，所述预设规则为当有偶数个相同的ID时，则删除所述偶数个相同的ID；
    当有奇数个相同的ID时，则只保留所述奇数个相同ID中的一个。
  15. 如权利要求14所述设备，其特征在于，所述第一判断模块，具体用于：
    在预设时间内判断接收的比较结果中相同ID的个数，如果相同ID的个数为偶数个，所述服务器删除所述偶数个相同的ID，当所述服务器中不存在ID时则表示所述用户发送的数据信息处理完成；
    如果相同ID的个数为奇数个，所述服务器保留所述奇数个相同ID中的一个，并继续接收携带有比较结果的信息，当在预定时间内所述服务器中还存在ID时则表示所述用户发送的数据没有处理完成，直到所述预设时间结束前所述服务器中不存在ID时通知所述用户其发送的数据信息处理完成。
  16. 如权利要求13所述设备，其特征在于，所述设备还包括：
    发送模块，在用户向逻辑节点发送数据信息后，用于向所述逻辑节点发送与所述用户发送的相同的数据信息；
    第二接收模块，用于接收所述逻辑节点返回的携带有比较结果的信息；
    第二判断模块，用于在预设时间内根据预设规则对所述信息中的比较结果进行判断，以使所述服务器根据判断结果判断所述用户发送的数据信息是否处理完成。
PCT/CN2016/084595 2015-06-12 2016-06-03 一种消息通知方法和设备 Ceased WO2016197858A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510325555.8A CN106302621B (zh) 2015-06-12 2015-06-12 一种消息通知方法和设备
CN201510325555.8 2015-06-12

Publications (1)

Publication Number Publication Date
WO2016197858A1 true WO2016197858A1 (zh) 2016-12-15

Family

ID=57502974

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/084595 Ceased WO2016197858A1 (zh) 2015-06-12 2016-06-03 一种消息通知方法和设备

Country Status (2)

Country Link
CN (1) CN106302621B (zh)
WO (1) WO2016197858A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112953835A (zh) * 2021-01-18 2021-06-11 广州三七互娱科技有限公司 数据传输方法、装置及系统
CN113590036A (zh) * 2021-07-26 2021-11-02 浪潮云信息技术股份公司 一种写入batch group串行化逻辑的优化方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110555038A (zh) * 2018-03-28 2019-12-10 阿里巴巴集团控股有限公司 一种数据处理系统、方法及装置
CN113946426B (zh) * 2021-10-29 2025-03-25 广州文远知行科技有限公司 一种任务处理方法、装置、电子设备和存储介质
CN114328129B (zh) * 2022-01-17 2025-08-12 蚂蚁区块链科技(上海)有限公司 消息发送方法、装置、设备及存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101887367A (zh) * 2010-06-22 2010-11-17 天津大学 一种多级并行化编程方法
US20140089023A1 (en) * 2012-09-14 2014-03-27 Tencent Technology (Shenzhen) Company Limited Method and system for distributing data stream

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102281134A (zh) * 2010-06-11 2011-12-14 中国移动通信集团公司 具有后继备份关系的逻辑节点的部属方法及设备
CN103916478B (zh) * 2014-04-11 2017-06-06 华为技术有限公司 基于分布式系统的流式构建数据方的方法和装置

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101887367A (zh) * 2010-06-22 2010-11-17 天津大学 一种多级并行化编程方法
US20140089023A1 (en) * 2012-09-14 2014-03-27 Tencent Technology (Shenzhen) Company Limited Method and system for distributing data stream

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MA, RUI.: "Design and Implementation of the Intercept and Prompting System on Fraud by SMS Based on the Storm", ELECTRONIC TECHNOLOGY & INFORMATION SCIENCE , CHINA MASTER'S THESES FULL-TEXT DATABASE, 15 April 2015 (2015-04-15), pages 1138 - 479, ISSN: 1674-0246 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112953835A (zh) * 2021-01-18 2021-06-11 广州三七互娱科技有限公司 数据传输方法、装置及系统
CN112953835B (zh) * 2021-01-18 2023-03-14 广州三七互娱科技有限公司 数据传输方法、装置及系统
CN113590036A (zh) * 2021-07-26 2021-11-02 浪潮云信息技术股份公司 一种写入batch group串行化逻辑的优化方法
CN113590036B (zh) * 2021-07-26 2024-03-29 上海沄熹科技有限公司 一种写入batch group串行化逻辑的优化方法

Also Published As

Publication number Publication date
CN106302621A (zh) 2017-01-04
CN106302621B (zh) 2019-11-08

Similar Documents

Publication Publication Date Title
Yang IoT stream processing and analytics in the fog
CN107807983B (zh) 一种支持大规模动态图数据查询的并行处理框架的设计方法
CN106980530A (zh) 一种分布式系统工作流处理方法和工作流引擎系统
CN110569252B (zh) 一种数据处理系统及方法
CN106712981B (zh) 一种节点变更通知方法及装置
WO2016197858A1 (zh) 一种消息通知方法和设备
CN104462037B (zh) 文档协同编辑中的一致性维护方法
CN107294801A (zh) 基于海量实时互联网dpi数据的流式处理方法及系统
US20220129408A1 (en) Data actor and data processing method thereof
CN103699447A (zh) 一个基于云计算的视频会议的转码分发系统
Ning et al. Mobile storm: Distributed real-time stream processing for mobile clouds
CN103793273A (zh) 一种基于Redis的分布式队列调度方法及装置
CN106354566B (zh) 一种命令处理的方法以及服务器
CN104077425B (zh) 一种基于操作转换的文本编辑实时协同方法
CN103942108B (zh) Hadoop同构集群下的资源参数优化方法
CN103716397B (zh) 一种面向服务仿真时钟推进方法
CN111831425A (zh) 一种数据处理方法、装置及设备
CN105354089A (zh) 支持迭代计算的流式数据处理模型及系统
Butakov et al. Multitenant approach to crawling of online social networks
Ke et al. Aggregation on the fly: reducing traffic for big data in the cloud
CN104935951A (zh) 一种基于分布式的视频转码方法
CN104702519A (zh) 流量卸载的方法、装置及系统
WO2023273157A1 (zh) 一种工作流生成方法、装置、设备及存储介质
CN116991562B (zh) 一种数据处理方法、装置、电子设备及存储介质
Han et al. Multi-tenancy-and redundancy-aware in-network aggregation using programmable switches

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16806756

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16806756

Country of ref document: EP

Kind code of ref document: A1