CN106487694B

CN106487694B - Data stream processing method and device

Info

Publication number: CN106487694B
Application number: CN201510534143.5A
Authority: CN
Inventors: 李旭良; 李嘉; 刘杰
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2015-08-27
Filing date: 2015-08-27
Publication date: 2020-03-27
Anticipated expiration: 2035-08-27
Also published as: WO2017032212A1; CN106487694A

Abstract

The embodiment of the invention discloses a data stream processing method and a device, wherein the method can comprise the following steps: a processing node of a real-time computing system receives a control flow which is sent by a management node of the real-time computing system and used for adjusting the topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure which needs to be updated currently by the real-time computing system; the processing node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node processes the data stream and sends the data stream after service processing according to the data stream circulation path included in the updated data stream circulation table. The embodiment of the invention can dynamically adjust the topological structure of the real-time computing system.

Description

Data stream processing method and device

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a method and an apparatus for processing a data stream.

Background

With the progress of science and technology and the development of user requirements, a large amount of real-time data exists, wherein the real-time data has strong real-time performance and large data volume, and is not limited, and the real-time data is mainly calculated by a real-time computing system in a stream computing manner, for example: storm application. However, in the current real-time computing system, since all the real-time computing systems depend on JAVA Application Programming Interface (API) to implement codes and package submission tasks, the service logic of the real-time computing system that is already running is not variable, that is, the Topology (Topology) structure of the real-time computing system is not variable during running in the real-time computing system. However, there are many demands to dynamically adjust the processing logic of the real-time computing system, that is, to dynamically adjust the topology of the real-time computing system, so as to adapt to the business requirements. Therefore, how to dynamically adjust the topology of the real-time computing system is a technical problem which needs to be solved urgently at present.

Disclosure of Invention

The embodiment of the invention provides a data stream processing method and a data stream processing device, which can dynamically adjust the topological structure of a real-time computing system.

In a first aspect, an embodiment of the present invention provides a data stream processing method, including:

a processing node of a real-time computing system receives a control flow which is sent by a management node of the real-time computing system and used for adjusting the topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure which needs to be updated currently by the real-time computing system;

the processing node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated;

and when the processing node receives the data stream, the processing node processes the data stream and sends the data stream after service processing according to the data stream circulation path included in the updated data stream circulation table.

In a first possible implementation manner of the first aspect, the method further includes:

the processing node sends the updated data flow table to a shared storage node of the real-time computing system;

and when the processing node recovers the fault, the processing node acquires the updated data flow circulation table from the shared storage node and sends the data flow according to the updated data flow circulation table.

With reference to the first aspect or the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the method further includes:

the processing node feeds back an update result of the data flow table update to a source node of the real-time computing system, so that the source node sends the control flow to the management node when the update structure represents that the update fails;

and when the updating result shows that the updating fails, the processing node receives the control flow sent by the management node and updates a data flow circulation table according to the current topology structure needing to be updated described by the control flow.

With reference to the first aspect or the first possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the method further includes:

and the processing nodes feed back the updated result of the data flow circulation table to the output nodes of the real-time computing system, and the output nodes summarize the updated results fed back by all the processing nodes of the real-time computing system and output the summarized results.

In a second aspect, an embodiment of the present invention provides a data stream processing method, including:

a management node of a real-time computing system receives a control flow which is sent by a source node of the real-time computing system and used for adjusting the topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure which needs to be updated currently by the real-time computing system;

the management node sends the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow circulation table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow circulation table comprises a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node performs business processing on the data stream, and sends the data stream after the business processing according to a data stream circulation path included in the updated data stream circulation table.

In a first possible implementation manner of the second aspect, the method further includes:

the management node establishes and maintains connection with each processing node in the real-time computing system;

the management node sends the control flow to a processing node for performing service processing on a data flow, and the method comprises the following steps:

the management node sends the control flow to each of the processing nodes of the real-time computing system in a broadcast manner.

In a third aspect, an embodiment of the present invention provides a data stream processing method, including:

a source node of a real-time computing system acquires a control flow for adjusting the topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure which needs to be updated currently by the real-time computing system;

the source node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated;

the source node sends the control flow to a management node of the real-time computing system, so that the management node sends the control flow to a processing node for performing service processing on a data flow, and the processing node updates a data flow table according to the current topology structure needing to be updated, which is described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated;

and the source node receives a data stream, sends the data stream to the processing node according to a data stream circulation path included in a data stream circulation table updated by the source node, performs business processing on the data stream by the processing node, and sends the data stream after the business processing according to the data stream circulation path included in the data stream circulation table updated by the processing node.

In a first possible implementation manner of the third aspect, the method further includes:

the source node sends the updated data flow circulation table to a shared storage node of the real-time computing system;

and when the source node recovers the fault, the source node acquires the updated data flow circulation table from the shared storage node and sends the data flow according to the updated data flow circulation table.

With reference to the third aspect or the first possible implementation manner of the third aspect, in a second possible implementation manner of the third aspect, the method further includes:

the source node receives an updating result of the processing node for updating the data flow table, wherein the updating result is fed back by the processing node;

when the updating result shows that the updating fails, the source node sends the control flow to the management node, so that the management node sends the control flow to the processing node, and the processing node updates the data flow circulation table according to the current topology structure needing updating described by the control flow.

In a fourth aspect, an embodiment of the present invention provides a data stream processing apparatus, where the apparatus is applied to a processing node of a real-time computing system, and the apparatus includes: receiving unit, first update unit and first sending unit, wherein:

the receiving unit is used for receiving a control flow which is sent by a management node of a real-time computing system and used for adjusting the topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure which needs to be updated currently by the real-time computing system;

the first updating unit is configured to update a data flow table according to the topology structure described by the control flow and currently needing to be updated, where the updated data flow table includes a data flow path matching the topology structure currently needing to be updated;

and the first sending unit is configured to, when the processing node receives a data stream, perform service processing on the data stream, and send the data stream after the service processing according to a data stream flow path included in the updated data stream flow table.

In a first possible implementation manner of the fourth aspect, the apparatus further includes:

the second sending unit is used for sending the updated data flow circulation table to a shared storage node of the real-time computing system;

and the recovery unit is used for acquiring the updated data stream flow table from the shared storage node and sending the data stream according to the updated data stream flow table when the processing node recovers the fault.

With reference to the fourth aspect or the first possible implementation manner of the fourth aspect, in a second possible implementation manner of the fourth aspect, the apparatus further includes:

a first feedback unit, configured to feed back an update result of updating the data flow table to a source node of the real-time computing system, so that when the update structure indicates that the update fails, the source node sends the control flow to the management node;

and the second updating unit is used for receiving the control flow sent by the management node when the updating result shows that the first updating unit fails to update, and updating a data flow circulation table according to the topology structure which is described by the control flow and currently needs to be updated.

With reference to the fourth aspect or the first possible implementation manner of the fourth aspect, in a third possible implementation manner of the fourth aspect, the apparatus further includes:

and the second feedback unit is used for feeding back the updated updating result of the data flow table to the output node of the real-time computing system, summarizing the updating result fed back by all the processing nodes of the real-time computing system by the output node, and outputting a summarizing result.

In a fifth aspect, an embodiment of the present invention provides a data stream processing apparatus, including: a receiving unit and a transmitting unit, wherein:

the receiving unit is used for receiving a control flow which is sent by a source node of a real-time computing system and used for adjusting the topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure which needs to be updated currently by the real-time computing system;

the sending unit is configured to send the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow circulation table according to the current topology structure needing to be updated, which is described by the control flow, where the updated data flow circulation table includes a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node performs business processing on the data stream, and sends the data stream after the business processing according to a data stream circulation path included in the updated data stream circulation table.

In a first possible implementation manner of the fifth aspect, the apparatus further includes:

a holding unit for establishing and holding connections with each of the processing nodes in the real-time computing system;

the sending unit is configured to send the control flow to each of the processing nodes of the real-time computing system in a broadcast manner.

In a sixth aspect, an embodiment of the present invention provides a data stream processing apparatus, where the apparatus is applied to a source node of a real-time computing system, and the apparatus includes: the device comprises an acquisition unit, an updating unit, a first sending unit and a second sending unit, wherein:

the acquiring unit is used for acquiring a control flow for adjusting the topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure which needs to be updated currently by the real-time computing system;

the updating unit is configured to update a data flow table according to the topology structure described by the control flow and currently needing to be updated, where the updated data flow table includes a data flow path matching the topology structure currently needing to be updated;

the first sending unit is configured to send the control flow to a management node of the real-time computing system, so that the management node sends the control flow to a processing node for performing service processing on a data flow, and the processing node updates a data flow table according to the current topology structure, described by the control flow, that needs to be updated, where the updated data flow table includes a data flow circulation path matched with the current topology structure that needs to be updated;

the second sending unit is configured to receive a data stream, send the data stream to the processing node according to a data stream circulation path included in the data stream circulation table updated by the source node, perform service processing on the data stream by the processing node, and send the data stream after the service processing according to the data stream circulation path included in the data stream circulation table updated by the processing node.

In a first possible implementation manner of the sixth aspect, the apparatus further includes:

a third sending unit, configured to send the updated data flow distribution table to a shared storage node of the real-time computing system;

and the recovery unit is used for acquiring the updated data stream flow table from the shared storage node when the source node recovers the fault, and sending the data stream according to the updated data stream flow table.

With reference to the sixth aspect or the first possible implementation manner of the sixth aspect, in a second possible implementation manner of the sixth aspect, the apparatus further includes:

a receiving unit, configured to receive an update result of the processing node updating the data flow table, where the update result is fed back by the processing node;

and a fourth sending unit, configured to send the control flow to the management node when the update result indicates that the update fails, so that the management node sends the control flow to the processing node, and the processing node updates the data flow table according to the topology structure, described by the control flow, that needs to be updated currently.

In the above technical solution, a processing node of a real-time computing system receives a control flow sent by a management node of the real-time computing system and used for adjusting a topology structure of the real-time computing system, where the control flow is used for describing the topology structure of the real-time computing system that needs to be updated currently; the processing node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node processes the data stream and sends the data stream after service processing according to the data stream circulation path included in the updated data stream circulation table. Therefore, the processing logic of the real-time computing system can be updated in a mode of updating the data flow circulation table in real time, namely the topological structure of the real-time computing system can be dynamically adjusted.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is an architecture diagram of a real-time computing system to which the data stream processing method provided by the embodiments of the present invention is applicable;

fig. 2 is a schematic flow chart of a data stream processing method according to an embodiment of the present invention;

fig. 3 is a schematic flow chart of another data stream processing method according to an embodiment of the present invention;

fig. 4 is a schematic flow chart of another data stream processing method according to an embodiment of the present invention;

fig. 5 is a schematic flow chart of another data stream processing method according to an embodiment of the present invention;

FIG. 6 is a control flow transmission diagram, for example, illustrating a Storm application, provided by an embodiment of the present invention;

FIG. 7 is a diagram illustrating data flow transmission in a Storm application according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a data stream processing apparatus according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 13 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 14 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 15 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 16 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 17 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 18 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention;

fig. 19 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, fig. 1 is an architecture diagram of a real-time computing system to which the data stream processing method according to the embodiment of the present invention is applicable, as shown in fig. 1, including: the monitoring system comprises a monitoring main body 11, a source node 12, one or more processing nodes 13 and a management module 14, wherein the monitoring main body 11 is respectively connected with the source node 12 and each processing node 13, the source node 12 is connected with at least one processing node 13, and each processing node 13 is connected in sequence. In addition, the management module 14 includes a management node 141 and an output node 142, wherein the management node 141 is connected to the source node and each processing node 13, and the output node 142 is connected to each processing node 13. The connection relationship between the nodes in the real-time computing system is represented by a Topology (Topology) of the real-time computing system, or understood as a Topology model.

It should be noted that the connection between the nodes described in this embodiment may be understood as a connection on a logical connection, and data flow or control flow may be transmitted between the connected nodes.

When the real-time computing system is in operation, a data stream flows in from a source node 12, then the source node 12 sends the data stream to a corresponding processing node 13 according to a stored data stream flow table, and the processing node 13 performs service processing on the received data stream, where the service processing may include performing real-time computation on the data stream or understanding as stream computation, the processing node 13 sends the service-processed data stream to another processing node 13 according to the stored data stream flow table, and the processing node 13 sends the service-processed data stream to another node or outputs a result according to the stored data stream flow table. In the embodiment of the present invention, the data flow table includes a data flow path matched with a topology structure of the real-time computing system, that is, in the embodiment, a transmission path or a transmission structure of a data flow is controlled by the data flow table, so that both the source node 12 and the processing node 13 can send a data flow that needs to be currently sent to a corresponding node through the data flow table stored in each of the source node and the processing node. For example: the topology of the real-time computing system includes a source node 12 connected to a processing node a, the processing node a is connected to a processing node B, the processing node B is connected to a processing node C, and the processing node C is an end node. Thus, the data flow table stored in the source node 12 may include a mapping for the data flow sent from the source node 12 to the processing node a, the data flow table stored in the processing node a may include a mapping for the data flow sent from the processing node a to the processing node B, the data flow table stored in the processing node B may include a mapping for the data flow sent from the processing node B to the processing node C, and the data flow table stored in the processing node C may include a mapping for the processing node C to output a service processing result. Of course, in this embodiment, the data flow table stored in each node may include all mappings or only mappings associated with itself.

While the real-time computing system is running, a control flow may be transmitted from source node 12 to management node 141, where the control flow is used to describe the topology of the real-time computing system that currently needs to be updated. The management node 141 then transmits the control flow to each processing node 13, so that the processing nodes 13 can update the stored data flow table according to the control flow, thereby dynamically adjusting the topology of the real-time computing system.

In addition, in the embodiment of the present invention, the real-time computing system may be a distributed system, that is, each node may be distributed and run in different machines, and of course, some of the nodes may also be allowed to run in the same machine, for example: the management node 121 and the output node 122 may run in the same machine or in different machines. In addition, in the embodiments of the present invention, these machines are not limited, for example: the machines may be computers or servers, etc.

Referring to fig. 2, fig. 2 is a schematic flow chart of a data stream processing method according to an embodiment of the present invention, as shown in fig. 2, including the following steps:

201. the method comprises the steps that a processing node of a real-time computing system receives a control flow which is sent by a management node of the real-time computing system and used for adjusting the topology structure of the real-time computing system, wherein the control flow is used for describing the topology structure which needs to be updated currently by the real-time computing system.

In this embodiment, the topology described by the control flow that needs to be updated currently may be a complete topology that needs to be updated currently by the real-time computing system, for example: the control flow described above may describe the topology between the source node, processing node, management node and output node as shown in fig. 1. Alternatively, the topology described by the control flow that needs to be updated currently may be the topology that needs to be adjusted currently, for example: in the original topology structure, processing node a is connected to processing node B, and processing node B is connected to processing node C, that is, the flow sequence of the data stream is that processing node a is connected to processing node B and then to processing node C, and the current flow sequence needs to be adjusted to processing node a is connected to processing node C, and processing node C is connected to processing node B. Then, the above control flow may only describe the topology of the processing node a connecting to the processing node C, and the processing node C then connecting to the processing node B, that is, after updating, the data flow flows in the sequence from the processing node a to the processing node B and then to the processing node C.

In addition, in the present embodiment, the control flow may be understood as control flow information, that is, the control flow may be understood as one piece of information.

202. And the processing node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated.

In this embodiment, the processing node stores a data flow table matched with the original topology structure, and when the control flow is received, that is, when the current topology structure needs to be updated, the data flow table can be updated according to the current topology structure that needs to be updated. The updated data flow table comprises a data flow circulation path matched with the current topological structure to be updated, so that the processing node can transmit the data flow according to the updated topological structure when transmitting the data flow.

The data flow distribution path matching the topology currently required to be updated may be a distribution path or a distribution structure of the data flow in the topology currently required to be updated. For example: the topology currently to be updated is processing node a connected to processing node C, and the updated data flow table includes the data flow transmitted from processing node a to processing node C, that is, when the processing node executing step 202 is processing node a, the updated data flow table includes the data flow path for transmitting the data flow to processing node C.

203. And when the processing node receives the data stream, the processing node processes the data stream and sends the data stream after service processing according to the data stream circulation path included in the updated data stream circulation table.

Through the steps, the processing nodes can transmit the data stream according to the updated topological structure, so that the topological structure of the real-time computing system can be dynamically adjusted. In addition, when the real-time computing system runs in a huge amount of business, the topological structure of the real-time computing system is dynamically adjusted through the steps, and interference on the data stream being processed is avoided. In addition, when the real-time computing system is a distributed system and each node runs in different machines, the problem caused by modification delay of a certain machine can be avoided by dynamically adjusting the topological structure of the real-time computing system through the steps.

In this embodiment, the processing node may be any processing node in a real-time computing system.

In this embodiment, a processing node of a real-time computing system receives a control flow sent by a management node of the real-time computing system and used for adjusting a topology structure of the real-time computing system, where the control flow is used to describe the topology structure that the real-time computing system currently needs to update; the processing node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node processes the data stream and sends the data stream after service processing according to the data stream circulation path included in the updated data stream circulation table. Therefore, the processing logic of the real-time computing system can be updated in a mode of updating the data flow circulation table in real time, namely the topological structure of the real-time computing system can be dynamically adjusted.

Referring to fig. 3, fig. 3 is a schematic flow chart of another data stream processing method according to an embodiment of the present invention, as shown in fig. 3, including the following steps:

301. the method comprises the steps that a processing node of a real-time computing system receives a control flow which is sent by a management node of the real-time computing system and used for adjusting the topology structure of the real-time computing system, wherein the control flow is used for describing the topology structure which needs to be updated currently by the real-time computing system.

302. And the processing node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated.

In this embodiment, when updating the data flow circulation table, the processing node may suspend data flow transmission, but may perform service processing on the data flow. Or the processing node may suspend the service processing and data stream transmission on the data stream when updating the data stream flow table. And when the data flow table is updated, resuming the suspended data flow transmission and/or the service processing of the data flow. This allows for dynamic adjustment of the topology of the real-time computing system, which would otherwise occur due to mishandling of the data stream and system congestion, affecting performance.

303. And when the processing node receives the data stream, the processing node processes the data stream and sends the data stream after service processing according to the data stream circulation path included in the updated data stream circulation table.

The above method may further comprise the steps of:

304. and the processing node sends the updated data flow circulation table to a shared storage node of the real-time computing system.

305. And when the processing node recovers the fault, the processing node acquires the updated data flow circulation table from the shared storage node and sends the data flow according to the updated data flow circulation table.

In this embodiment, it may be implemented that, after a failure occurs, the processing node may directly obtain the data flow table updated in step 302 from the shared storage node when performing recovery, so that, after a failure occurs, the processing node may read and initialize the data flow table from the shared storage node, that is, a High Availability (HA) mechanism for dynamically adjusting the topology is completed.

In this embodiment, the method may further include the following steps:

In this embodiment, the update result may be the update result of step 302, and when the update of step 302 is successful, the source node may be informed of the success of the update of the processing node through the update result. When the updating fails, the source node is informed of the failure of the updating of the processing node through the updating result, so that the source node can send the control flow to the management node again, and the management node sends the control flow to the processing node again to enable the processing node to update again. Certainly, when the source node sends the control flow to the management node again, the source node may also carry the identification information of the processing node that fails to be updated, so that the management node can only send the control flow to the processing node that fails to be updated again, and does not send the control flow to the processing node that succeeds in being updated again, so as to save transmission resources.

The embodiment can realize that the updating result can be correctly fed back, and if the updating fails, the updating task can be started again.

In this embodiment, the method may further include the following steps:

The output node may obtain the update result fed back by each processing node, so that the output node may summarize the update results, and may output the summarized result, for example: and sending the summary result to a display device, or printing the summary result, so that a user knows the state of the topological structure adjustment of the real-time computing system.

In this embodiment, various optional implementations are added to the embodiment shown in fig. 2, and the topology of the real-time computing system can be dynamically adjusted.

Referring to fig. 4, fig. 4 is a schematic flow chart of another data stream processing method according to an embodiment of the present invention, as shown in fig. 4, including the following steps:

401. the method comprises the steps that a management node of a real-time computing system receives a control flow which is sent by a source node of the real-time computing system and used for adjusting the topology structure of the real-time computing system, wherein the control flow is used for describing the topology structure which needs to be updated currently by the real-time computing system.

In this embodiment, the control flow may include structural information of a topology structure that needs to be updated currently, and may further include a control flow decomposition structure identifier agreed by internal processing logic of the management node or the management module, that is, the topology structure that needs to be updated currently may be identified through the control flow decomposition structure identifier and the structural information.

402. The management node sends the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow circulation table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow circulation table comprises a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node performs business processing on the data stream, and sends the data stream after the business processing according to a data stream circulation path included in the updated data stream circulation table.

In this embodiment, the management node may send the control flow to all processing nodes in the real-time computing system, so that all processing nodes update the data flow tables stored in the processing nodes. For example: the above method may further comprise the steps of:

the step of sending the control flow to a processing node for performing service processing on a data flow by the management node may include:

This allows the control flow to be broadcast to all processing nodes.

Of course, in this embodiment, the management node may also send a control flow to only a part of the processing nodes, for example: when only part of the processing nodes are involved in the topology that needs to be updated currently, the management node may send control flows only to the involved processing nodes, so that the involved processing nodes update the data flow circulation table, and the management node may not update the data flow circulation table if the involved processing nodes are not involved.

In this embodiment, a management node of a real-time computing system receives a control flow sent by a source node of the real-time computing system and used for adjusting a topology structure of the real-time computing system, where the control flow is used to describe the topology structure that the real-time computing system currently needs to update; the management node sends the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow circulation table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow circulation table comprises a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node performs business processing on the data stream, and sends the data stream after the business processing according to a data stream circulation path included in the updated data stream circulation table. This may enable dynamic adjustment of the topology of the real-time computing system.

Referring to fig. 5, fig. 5 is a schematic flow chart of another data stream processing method according to an embodiment of the present invention, as shown in fig. 5, including the following steps:

501. a source node of a real-time computing system obtains a control flow for adjusting a topology of the real-time computing system, wherein the control flow is used for describing the topology of the real-time computing system which needs to be updated currently.

The control flow may be a control flow for the source node to receive user input. The control flow here may refer to the control flow described in the embodiment shown in fig. 1 to 4, and will not be described repeatedly here.

502. And the source node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated.

After receiving the control flow, the source node can decompose the control flow and update the data flow table according to the topology structure which is obtained by decomposition and needs to be updated currently. In addition, the source node stores or caches the updated data flow table. Of course, the control flow may also be stored or cached.

503. And the source node sends the control flow to a management node of the real-time computing system, so that the management node sends the control flow to a processing node for performing service processing on a data flow, and the processing node updates a data flow table according to the current topology needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology needing to be updated.

Through the steps, the management node can send the control flow to the processing node, and the processing node updates the data flow circulation table stored by the processing node.

504. And the source node receives the data stream, sends the data stream to the processing node according to the data stream circulation path included in the data stream circulation table updated by the source node, performs business processing on the data stream by the processing node, and sends the data stream after the business processing according to the data stream circulation path included in the data stream circulation table updated by the processing node.

In this embodiment, the data stream may be data input by a user or data transmitted by a collection device, and the like, which are currently required to be calculated by a real-time computing system.

Since the data flow table is updated in step 502, step 504 sends the data flow according to the updated data flow table, so that the source node can send the data flow according to the updated topology structure, thereby dynamically adjusting the topology structure of the real-time computing system.

In this embodiment, the method may further include the following steps:

In this embodiment, it can be realized that the data flow table updated in step 502 can be directly obtained from the shared storage node when the source node recovers after a failure occurs, so that it can be realized that the data flow table can be read from the shared storage node and initialized after the source node fails, that is, the HA mechanism for dynamically adjusting the topology structure is completed.

In this embodiment, the method may further include the following steps:

In this embodiment, the update result of the current update data flow table of each processing node can be obtained in time. When one or more processing nodes fail to update, the source node may trigger the management node to send a control flow to the processing node that fails to update, so that the processing node that fails to update updates the data flow circulation table again.

In this embodiment, a source node of a real-time computing system obtains a control flow for adjusting a topology of the real-time computing system, where the control flow is used to describe a topology that the real-time computing system currently needs to update; the source node updates a data flow table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated; the source node sends the control flow to a management node of the real-time computing system, so that the management node sends the control flow to a processing node for performing service processing on a data flow, and the processing node updates a data flow table according to the current topology structure needing to be updated, which is described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated; and the source node receives a data stream, sends the data stream to the processing node according to a data stream circulation path included in a data stream circulation table updated by the source node, performs business processing on the data stream by the processing node, and sends the data stream after the business processing according to the data stream circulation path included in the data stream circulation table updated by the processing node. The topology of the real-time computing system may be dynamically adjusted.

Referring to fig. 6, fig. 6 is a schematic diagram of control flow transmission by taking Storm application as an example according to an embodiment of the present invention, as shown in fig. 6, including:

601. and the alarm node receives the control flow message, analyzes the control flow message to obtain a control flow, updates a data flow table according to the control flow and sends the control flow message to the police station node.

Where Storm applications are real-time computing systems. The above-mentioned alert node may be understood as the source node in the embodiments shown in fig. 1-5. The alarm node is a Spout function node of the Storm real-time computing system, wherein the Spout function node is a data source class in Storm application and is used for receiving and sending external data streams or constructing data streams by the alarm node to send the data streams.

The police station nodes are understood as management nodes in the embodiments shown in fig. 1-5, the police station nodes are Bolt function type nodes in Storm application, rely on the support of a police module main body (the management module main body in the embodiment shown in fig. 1), are internally and automatically integrated in a Storm application topological structure, and are communicated with all nodes except alarm nodes sending data to the police station nodes in the Storm application. The Bolt functional class nodes are data processing nodes in Storm application, each Bolt realizes different service logics, and a plurality of bolts are combined to complete complex service logic processing.

602. The police station node sends the control flow message to the police node A, the police node B and the police node C.

The police node may be understood as a processing node in the embodiments shown in fig. 1 to fig. 5, and is a Bolt function class node.

603. And the police node A decomposes the control flow message and updates the data flow table according to the control flow obtained by decomposition.

604. The police node A feeds back the control flow processing message to the alarm node, and feeds back the feedback message of the processing process to the alarm node.

When the update fails in step 603, the police node a may obtain the control flow message from the alarm node to implement the update again, in this case, the control flow processing message fed back to the alarm node may be an update result of the update again.

605. And the police node B decomposes the control flow message and updates the data flow table according to the control flow obtained by decomposition.

606. And the police node B feeds the control flow processing message back to the alarm node, and feeds the feedback message of the processing process back to the alarm node.

607. And the police node C decomposes the control flow message and updates the data flow table according to the control flow obtained by decomposition.

608. And the police node C feeds back the control flow processing message to the alarm node, and feeds back the feedback message of the processing process to the alarm node.

609. The alert nodes aggregate the updated structure of the feedback and send or print the control flow processing structure according to a predetermined logic.

Referring to fig. 7, fig. 7 is a schematic diagram of data stream transmission taking Storm application as an example according to an embodiment of the present invention, as shown in fig. 7, including:

701. and the alarm node receives the data stream and sends the data stream to the processing node A according to the data stream flow table.

702. And the police node A processes the received data stream and sends the data stream after the service processing to the police node B according to the data stream flow table.

703. And the police node B performs business processing on the received data stream and sends the data stream after the business processing to the police node C according to the data stream flow table.

704. And the police node C processes the received data stream and sends the data stream after the business processing to the next node or outputs a result according to the data stream flow table.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a data stream processing apparatus according to an embodiment of the present invention, as shown in fig. 8, including: a receiving unit 81, a first updating unit 82 and a first sending unit 83, wherein:

the receiving unit 81 is configured to receive a control flow sent by a management node of a real-time computing system and used for adjusting a topology of the real-time computing system, where the control flow is used to describe a topology that the real-time computing system currently needs to update.

It should be noted that the data stream processing apparatus in this embodiment may be applied to a processing node of a real-time computing system, for example: such as the processing nodes shown in fig. 1.

A first updating unit 82, configured to update a data flow table according to the topology described by the control flow, where the updated data flow table includes a data flow path matching the topology that needs to be updated currently.

The data flow distribution path matching the topology currently required to be updated may be a distribution path or a distribution structure of the data flow in the topology currently required to be updated. For example: the topology currently to be updated is processing node a connected to processing node C, and the updated data flow table includes the data flow transmitted from processing node a to processing node C, that is, when the device is applied to processing node a, the updated data flow table includes the data flow path for transmitting the data flow to processing node C.

A first sending unit 83, configured to, when the processing node receives a data flow, perform service processing on the data flow, and send the data flow after service processing according to a data flow distribution path included in the updated data flow distribution table.

The device can realize that the processing node transmits the data stream according to the updated topological structure, thereby realizing the dynamic adjustment of the topological structure of the real-time computing system. In addition, when the real-time computing system runs in a huge amount of business, the topological structure of the real-time computing system is dynamically adjusted through the steps, and interference on the data stream being processed is avoided. In addition, when the real-time computing system is a distributed system and each node runs in different machines, the problem caused by modification delay of a certain machine can be avoided by dynamically adjusting the topological structure of the real-time computing system through the steps.

Referring to fig. 9, fig. 9 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention, as shown in fig. 9, including: a receiving unit 91, a first updating unit 92 and a first sending unit 93, wherein:

the receiving unit 91 is configured to receive a control flow sent by a management node of the real-time computing system and used for adjusting the topology of the real-time computing system, where the control flow is used to describe the topology of the real-time computing system that needs to be updated currently.

A first updating unit 92, configured to update a data flow table according to the topology described by the control flow, where the updated data flow table includes a data flow path matching the topology that needs to be updated currently.

A first sending unit 93, configured to, when the processing node receives a data flow, perform service processing on the data flow, and send the data flow after service processing according to a data flow distribution path included in the updated data flow distribution table.

In this embodiment, the apparatus may further include:

a second sending unit 94, configured to send the updated data flow table to a shared storage node of the real-time computing system;

a recovering unit 95, configured to, when the processing node performs failure recovery, obtain the updated data flow distribution table from the shared storage node, and send a data flow according to the updated data flow distribution table.

In this embodiment, when the processing node fails and recovers, the data flow table updated by the first updating unit 92 may be directly obtained from the shared storage node, so that the processing node may read and initialize the data flow table from the shared storage node after the processing node fails, that is, the HA mechanism for dynamically adjusting the topology is completed.

As shown in fig. 10, the apparatus may further include:

a first feedback unit 96, configured to feed back an update result of updating the data flow table to a source node of the real-time computing system, so that when the update structure indicates that the update fails, the source node sends the control flow to the management node;

a second updating unit 97, configured to receive the control flow sent by the management node when the update result indicates that the first updating unit fails to update, and update the data flow table according to the topology described by the control flow and currently needing to be updated.

In this embodiment, the update result may be an update result of the first updating unit 92, and when the first updating unit 92 is updated successfully, the source node may be informed of the successful update of the processing node through the update result. When the updating fails, the source node is informed of the failure of the updating of the processing node through the updating result, so that the source node can send the control flow to the management node again, and the management node sends the control flow to the processing node again to enable the processing node to update again. Certainly, when the source node sends the control flow to the management node again, the source node may also carry the identification information of the processing node that fails to be updated, so that the management node can only send the control flow to the processing node that fails to be updated again, and does not send the control flow to the processing node that succeeds in being updated again, so as to save transmission resources.

As shown in fig. 11, the apparatus may further include:

the second feedback unit 98 is configured to feed back an update result of updating the data flow table to an output node of the real-time computing system, and the output node summarizes the update results fed back by all processing nodes of the real-time computing system and outputs a summarized result.

In this embodiment, various optional implementations are added to the embodiment shown in fig. 8, and the topology of the real-time computing system can be dynamically adjusted.

Referring to fig. 12, fig. 12 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention, as shown in fig. 12, including: a receiving unit 121 and a transmitting unit 122, wherein:

the receiving unit 121 is configured to receive a control flow sent by a source node of a real-time computing system to adjust a topology of the real-time computing system, where the control flow is used to describe a topology of the real-time computing system that needs to be updated currently.

It should be noted that the data stream processing apparatus provided in this embodiment may be applied to a management node of a real-time computing system, for example: the management node shown in fig. 1.

A sending unit 122, configured to send the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow table according to the current topology structure needing to be updated described in the control flow, where the updated data flow table includes a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node performs business processing on the data stream, and sends the data stream after the business processing according to a data stream circulation path included in the updated data stream circulation table.

In this embodiment, the management node may send the control flow to all processing nodes in the real-time computing system, so that all processing nodes update the data flow tables stored in the processing nodes. For example: as shown in fig. 13, the apparatus may further include:

a holding unit 123 for establishing and holding connections with the respective processing nodes in the real-time computing system;

the sending unit 122 is configured to send the control flow to each of the processing nodes of the real-time computing system in a broadcast manner.

This allows the control flow to be broadcast to all processing nodes.

Referring to fig. 14, fig. 14 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention, as shown in fig. 14, including: an obtaining unit 141, an updating unit 142, a first sending unit 143, and a second sending unit 144, wherein:

an obtaining unit 141, configured to obtain a control flow for adjusting the topology of the real-time computing system, where the control flow is used to describe the topology of the real-time computing system that needs to be updated currently;

it should be noted that the data stream processing apparatus provided in this embodiment may be applied to a source node of a real-time computing system, for example: such as the source node shown in fig. 1.

An updating unit 142, configured to update a data flow table according to the topology described by the control flow and currently needing to be updated, where the updated data flow table includes a data flow path matching the topology currently needing to be updated.

A first sending unit 143, configured to send the control flow to a management node of the real-time computing system, so that the management node sends the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow table according to the current topology needing to be updated described in the control flow, where the updated data flow table includes a data flow circulation path matching the current topology needing to be updated;

the unit can realize that the management node sends the control flow to the processing node, and the processing node updates the data flow circulation table stored by the processing node.

A second sending unit 144, configured to receive a data flow, send the data flow to the processing node according to a data flow circulation path included in the data flow circulation table updated by the source node, where the processing node performs service processing on the data flow, and send the data flow after service processing according to the data flow circulation path included in the data flow circulation table updated by the processing node.

Since the updating unit 142 updates the data flow circulation table, the second sending unit 144 sends the data flow according to the updated data flow circulation table, so that the source node can send the data flow according to the updated topology structure, thereby dynamically adjusting the topology structure of the real-time computing system.

In this embodiment, as shown in fig. 15, the apparatus may further include:

a third sending unit 145, configured to send the updated data flow table to a shared storage node of the real-time computing system;

a recovering unit 146, configured to, when the source node performs failure recovery, obtain the updated data flow table from the shared storage node, and send a data flow according to the updated data flow table.

In this embodiment, it may be achieved that the source node may directly obtain the data flow table updated by the updating unit 142 from the shared storage node when performing recovery after the failure occurs, so that the source node may read and initialize the data flow table from the shared storage node after the failure occurs, that is, the HA mechanism for dynamically adjusting the topology is completed.

In this embodiment, as shown in fig. 16, the apparatus may further include:

a receiving unit 147, configured to receive an update result of the processing node updating the data flow table, where the update result is fed back by the processing node;

a fourth sending unit 148, configured to send the control flow to the management node when the update result indicates that the update fails, so that the management node sends the control flow to the processing node, and the processing node updates the data flow table according to the topology described by the control flow and currently needing to be updated.

Referring to fig. 17, fig. 17 is a schematic structural diagram of another data stream processing apparatus applied to a processing node of a real-time computing system according to an embodiment of the present invention, as shown in fig. 17, including: a processor 171, a network interface 172, a memory 173 and a communication bus 174, wherein the communication bus 174 is used for realizing the connection communication among the processor 171, the network interface 172 and the memory 173, and the processor 171 executes the program stored in the memory 173 for realizing the following methods:

receiving a control flow which is sent by a management node of the real-time computing system and used for adjusting the topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure which needs to be updated currently by the real-time computing system;

updating a data flow table according to the topology structure which is described by the control flow and needs to be updated currently, wherein the updated data flow table comprises a data flow circulation path which is matched with the topology structure which needs to be updated currently;

and when the processing node receives the data stream, performing service processing on the data stream, and sending the data stream after the service processing according to a data stream circulation path included in the updated data stream circulation table.

In this embodiment, the processor 171 may further execute the following program:

sending the updated data flow table to a shared storage node of the real-time computing system;

and when the processing node recovers the fault, acquiring the updated data flow circulation table from the shared storage node, and sending the data flow according to the updated data flow circulation table.

feeding back an update result of the data flow table update to a source node of the real-time computing system, so that when the update structure represents that the update fails, the source node sends the control flow to the management node;

and feeding back the updated result of the data flow circulation table to an output node of the real-time computing system, summarizing the updated result fed back by all processing nodes of the real-time computing system by the output node, and outputting a summarized result.

Referring to fig. 18, fig. 18 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention, which is applied to a management node of a real-time computing system, and as shown in fig. 18, the apparatus includes: a processor 181, a network interface 182, a memory 183 and a communication bus 184, wherein the communication bus 184 is used for realizing the connection and communication among the processor 181, the network interface 182 and the memory 183, and the processor 181 executes the program stored in the memory 183 to realize the following methods:

receiving a control flow which is sent by a source node of the real-time computing system and used for adjusting the topology structure of the real-time computing system, wherein the control flow is used for describing the topology structure which needs to be updated currently by the real-time computing system;

sending the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow circulation table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow circulation table comprises a data flow circulation path matched with the current topology structure needing to be updated; and when the processing node receives the data stream, the processing node performs business processing on the data stream, and sends the data stream after the business processing according to a data stream circulation path included in the updated data stream circulation table.

In this embodiment, the processor 181 may further execute the following programs:

establishing and maintaining connections with each of the processing nodes in the real-time computing system;

the program executed by the processor 181 to send the control flow to the processing node for performing the service processing on the data flow may include:

Referring to fig. 19, fig. 19 is a schematic structural diagram of another data stream processing apparatus according to an embodiment of the present invention, which is applied to a source node of a real-time computing system, and as shown in fig. 19, the apparatus includes: a processor 191, a network interface 192, a memory 193 and a communication bus 194, wherein the communication bus 194 is used for realizing the connection and communication among the processor 191, the network interface 192 and the memory 193, and the processor 191 executes the program stored in the memory 193 for realizing the following method:

obtaining a control flow for adjusting the topology of the real-time computing system, wherein the control flow is used for describing the topology of the real-time computing system which needs to be updated currently;

sending the control flow to a management node of the real-time computing system, so that the management node sends the control flow to a processing node for performing service processing on a data flow, and the processing node updates a data flow table according to the current topology structure needing to be updated, which is described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated;

and receiving a data stream, sending the data stream to the processing node according to a data stream circulation path included in the data stream circulation table updated by the source node, performing service processing on the data stream by the processing node, and sending the data stream after the service processing according to the data stream circulation path included in the data stream circulation table updated by the processing node.

In this embodiment, the program executed by the processor 191 may further include:

and when the source node recovers the fault, acquiring the updated data flow circulation table from the shared storage node, and sending the data flow according to the updated data flow circulation table.

receiving an updating result of the processing node for updating the data flow table, which is fed back by the processing node;

and when the updating result shows that the updating fails, sending the control flow to the management node so that the management node sends the control flow to the processing node, and updating the data flow table by the processing node according to the current topology structure needing updating described by the control flow.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims

1. A method for processing a data stream, comprising:

a processing node of a real-time computing system receives a control flow which is sent by a management node of the real-time computing system and used for adjusting a topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure of the real-time computing system which needs to be updated currently, and comprises structural information of the topological structure which needs to be updated currently and a control flow decomposition structure identifier which is agreed by the internal processing logic of the management node or the management node;

when the processing node receives a data stream, the processing node processes the data stream and sends the data stream after service processing according to a data stream circulation path included in the updated data stream circulation table;

when the processing node updates the data flow circulation table, the data flow transmission is suspended and/or the service processing is carried out on the data flow; and when the processing node finishes updating the data flow circulation table, resuming the suspended data flow transmission and/or the service processing of the data flow.

2. The method of claim 1, wherein the method further comprises:

3. The method of claim 1 or 2, wherein the method further comprises:

the processing node feeds back an update result of the data flow table update to a source node of the real-time computing system, so that the source node sends the control flow to the management node when the update result represents that the update fails;

4. The method of claim 1 or 2, wherein the method further comprises:

5. A method for processing a data stream, comprising:

a management node of a real-time computing system receives a control flow which is sent by a source node of the real-time computing system and used for adjusting a topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure of the real-time computing system which needs to be updated currently, the control flow comprises structural information of the topological structure which needs to be updated currently and a control flow decomposition structure identifier which is agreed by internal processing logic of the management node or the management node;

the management node sends the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow circulation table according to the current topology structure needing to be updated described by the control flow, wherein the updated data flow circulation table comprises a data flow circulation path matched with the current topology structure needing to be updated; when the processing node receives a data stream, the processing node performs business processing on the data stream, and sends the data stream after the business processing according to a data stream circulation path included in the updated data stream circulation table; when the processing node updates the data flow circulation table, the data flow transmission is suspended and/or the service processing is carried out on the data flow; and when the processing node finishes updating the data flow circulation table, resuming the suspended data flow transmission and/or the service processing of the data flow.

6. The method of claim 5, wherein the method further comprises:

7. A method for processing a data stream, comprising:

a source node of a real-time computing system acquires a control flow for adjusting a topological structure of the real-time computing system, wherein the control flow is used for describing the topological structure of the real-time computing system which needs to be updated currently, and the control flow comprises structural information of the topological structure which needs to be updated currently and a control flow decomposition structure identifier which is agreed by a management node of the real-time computing system or internal processing logic of the management node;

the source node sends the control flow to the management node, so that the management node sends the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow table according to the current topology structure needing to be updated, which is described by the control flow, wherein the updated data flow table comprises a data flow circulation path matched with the current topology structure needing to be updated; when the processing node updates the data flow circulation table, the data flow transmission is suspended and/or the service processing is carried out on the data flow; when the processing node finishes updating the data flow circulation table, the suspended data flow sending and/or the service processing of the data flow are/is resumed;

8. The method of claim 7, wherein the method further comprises:

9. The method of claim 7 or 8, wherein the method further comprises:

10. A data stream processing apparatus, the apparatus being applied to a processing node of a real-time computing system, comprising: receiving unit, first update unit and first sending unit, wherein:

the receiving unit is configured to receive a control flow sent by a management node of the real-time computing system and used for adjusting a topology structure of the real-time computing system, where the control flow is used to describe the topology structure of the real-time computing system that needs to be updated currently, and the control flow includes structure information of the topology structure that needs to be updated currently and a control flow decomposition structure identifier agreed by processing logic inside the management node or the management node;

the first sending unit is configured to, when the processing node receives a data stream, perform service processing on the data stream, and send the data stream after the service processing according to a data stream flow path included in the updated data stream flow table;

when the first updating unit updates the data flow circulation table, the first sending unit suspends data flow sending and/or performs service processing on the data flow; and when the first updating unit finishes updating the data stream flow table, resuming the suspended data stream transmission and/or the service processing of the data stream.

11. The apparatus of claim 10, wherein the apparatus further comprises:

12. The apparatus of claim 10 or 11, wherein the apparatus further comprises:

a first feedback unit, configured to feed back an update result of updating the data flow table to a source node of the real-time computing system, so that when the update result indicates that updating fails, the source node sends the control flow to the management node;

13. The apparatus of claim 10 or 11, wherein the apparatus further comprises:

14. A data stream processing apparatus, comprising: a receiving unit and a transmitting unit, wherein:

the receiving unit is configured to receive a control flow sent by a source node of a real-time computing system and used for adjusting a topology structure of the real-time computing system, where the control flow is used for describing the topology structure of the real-time computing system that needs to be updated currently, and the control flow includes structure information of the topology structure that needs to be updated currently and a control flow decomposition structure identifier agreed by a management node of the real-time computing system or internal processing logic of the management node;

the sending unit is configured to send the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow circulation table according to the current topology structure needing to be updated, which is described by the control flow, where the updated data flow circulation table includes a data flow circulation path matched with the current topology structure needing to be updated; when the processing node receives a data stream, the processing node performs business processing on the data stream, and sends the data stream after the business processing according to a data stream circulation path included in the updated data stream circulation table; when the processing node updates the data flow circulation table, the data flow transmission is suspended and/or the service processing is carried out on the data flow; and when the processing node finishes updating the data flow circulation table, resuming the suspended data flow transmission and/or the service processing of the data flow.

15. The apparatus of claim 14, wherein the apparatus further comprises:

16. A data stream processing apparatus applied to a source node of a real-time computing system, comprising: the device comprises an acquisition unit, an updating unit, a first sending unit and a second sending unit, wherein:

the acquiring unit is configured to acquire a control flow used for adjusting a topology structure of the real-time computing system, where the control flow is used for describing the topology structure of the real-time computing system that needs to be updated currently, and the control flow includes structure information of the topology structure that needs to be updated currently and a control flow breakdown structure identifier agreed by a management node of the real-time computing system or internal processing logic of the management node;

the first sending unit is configured to send the control flow to the management node, so that the management node sends the control flow to a processing node for performing service processing on a data flow, so that the processing node updates a data flow table according to the current topology structure needing to be updated, which is described by the control flow, where the updated data flow table includes a data flow circulation path matched with the current topology structure needing to be updated; when the processing node updates the data flow circulation table, the data flow transmission is suspended and/or the service processing is carried out on the data flow; when the processing node finishes updating the data flow circulation table, the suspended data flow sending and/or the service processing of the data flow are/is resumed;

17. The apparatus of claim 16, wherein the apparatus further comprises:

18. The apparatus of claim 16 or 17, wherein the apparatus further comprises: