CN104267939A

CN104267939A - Business processing method, device and system

Info

Publication number: CN104267939A
Application number: CN201410474486.2A
Authority: CN
Inventors: 方丰斌; 葛庆环
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2014-09-17
Filing date: 2014-09-17
Publication date: 2015-01-07
Anticipated expiration: 2034-09-17
Also published as: CN104267939B

Abstract

The invention discloses a business processing method. A working node acquires operators in a stream-oriented computation application model dispatched by a master node, the operators in the stream-oriented computation application model and data stream directions among the operators form at least one iteration loop, iteration times corresponding to each iteration loop is configured on an outlet operator of each iteration loop, the outlet operator is the last operator in the data stream direction of each iteration loop and receives business data, and the business data are processed according to a partial processing logic carried by each operator and are outputted from the outlet operator when the fact that the iteration times of the business data reaches that of the iteration loop with each outlet operator is determined according to the partial processing logic carried of each outlet operator. By the business processing method, data processing delay can be reduced, and data processing instantaneity is improved.

Description

A kind of method of business processing, Apparatus and system

Technical field

The present invention relates to field of computer technology, be specifically related to a kind of method of business processing, Apparatus and system.

Background technology

Real-time streams calculates when referring to that data occur and just processes it at once, an event occurs and carries out single treatment, instead of be cached batch processing, supports that the computing system that real-time streams calculates is called stream calculation system or stream calculation platform.

In stream calculation system, business processing logic needs to be converted into streaming computing application model usually, streaming computing application model is generally without loop digraph (Directed Acyclic Graph, DAG), also referred to as application flow graph, as shown in Figure 1, streaming computing application model is formed primarily of operator (operator) and stream (stream), wherein operator (operator) is the data processing unit of bearer service logic, can by stream calculation platform distributed scheduling according to minimum unit, flowing (stream) is then data mutual between operator.

Iterative processing is the computing method that a kind of old value recursion of continuous variable is newly worth.Iterative processing is widely used at Data Mining.Along with the rise and development that social networks, knowledge mapping, biological information network etc. are applied, the data scale of iterative processing expands day by day, and data structure is more complicated, and algorithm Time and place expense is faced with larger challenge.

In prior art when carrying out iterative processing, all need to store data advanced person row, and then process in batches, cause the time delay of data processing higher, real-time is poor.

Summary of the invention

In order to solve, iterative processing time delay in prior art is high, the problem of poor real, the method of a kind of business processing that the embodiment of the present invention provides, the data of the business received directly are carried out iterative processing, do not need first to store, the processing delay of the data of business can be reduced, improve real-time.The embodiment of the present invention additionally provides corresponding Apparatus and system.

First aspect present invention provides a kind of method of business processing, described method is applied to the working node in stream calculation system, described stream calculation system also comprises main controlled node, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business, described method comprises:

Described working node obtains the operator of described main controlled node scheduling, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and the outlet operator of each iteration loop is configured with iterations corresponding to described each iteration loop, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop,

Receive the data of described business;

Process according to the data of part processing logic to described business that described each operator carries, at the part processing logic according to each outlet operator, when determining to have reached the iterations of the iteration loop at described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

In conjunction with first aspect, in the implementation that the first is possible, after the data of the described business of described reception, described method also comprises:

For described each iteration loop arranges the corresponding flag of iterations in the data of described business;

The described data of part processing logic to described business carried according to described each operator process, at the part processing logic according to each outlet operator, when determining to have reached the iterations of the iteration loop at described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator, specifically comprise:

For each iteration loop, when the data of described business carry out an iterative processing in described each iteration loop, the flag of the outlet operator place of the described each iteration loop iterations corresponding to each iteration loop described adds 1;

When the iterations corresponding with each iteration loop described when the iterations indicated by the flag of iterations that each iteration loop described is corresponding is equal, export the data of the described business after each iteration loop iterative processing described from the outlet operator of each iteration loop described.

In conjunction with the first possible implementation of first aspect, in the implementation that the second is possible, after the described outlet operator from each iteration loop described exports the data of the described business after each iteration loop iterative processing described, described method also comprises:

The iterations of the flag of iterations corresponding for each iteration loop described is made zero.

In conjunction with the implementation that first aspect, first aspect the first or the second are possible, in the implementation that the third is possible, described method also comprises:

According to the processing logic receiving notice message of source operator, described source operator is that in described streaming computing application model, data stream walks first operator upwards, comprise the routing information arriving incremental computations operator from described source operator in described notification message, described incremental computations operator is the operator participating in incremental computations;

According to described routing information, described notification message is delivered to described incremental computations operator by described source operator, described notification message is that described main controlled node sends when arriving the triggered time of described incremental computations operator, and the triggered time of described incremental computations operator is: described timer completes the timing timing statistics point to incremental computations time value corresponding to described incremental computations operator;

According to described participation incremental computations operator part processing logic, incremental computations is carried out to the data of described business.

In conjunction with the third possible implementation of first aspect, in the 4th kind of possible implementation, described according to described participation incremental computations operator part processing logic, incremental computations is carried out to the data of described business, specifically comprises:

According to described participation incremental computations operator part processing logic, carry out data queue's switching, suspend the process of the data to the described business that the described triggered time receives, the data of the described business that priority processing received before the described triggered time.

Second aspect present invention provides a kind of method of business processing, and described method is applied to the main controlled node in stream calculation system, and described stream calculation system also comprises working node, and described method comprises:

Described main controlled node obtains the streaming computing application model of business, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and each iteration loop is configured with corresponding iterations, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop,

Be configured on the outlet operator of described every iteration loop by the iterations of described each iteration loop correspondence, described outlet operator is that in described each iteration loop, data stream walks last operator upwards;

By each Operator Scheduling in the described streaming computing application model after configuration on described working node, processed by the data of part processing logic to described business carried according to described each operator to make described working node, according to the part processing logic of each outlet operator, when determining the iterations of the iteration loop having reached described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

In conjunction with second aspect, in the implementation that the first is possible, described method also comprises:

Described main controlled node determines incremental computations operator in described streaming computing application model, and the incremental computations time value of described incremental computations operator, and described incremental computations operator is the operator participating in incremental computations;

The incremental computations time value of described incremental computations operator is added in the time triggered queue of timer;

For each incremental computations operator, when arriving the triggered time of each incremental computations operator described, determine the routing information arriving each incremental computations operator described from source operator, the triggered time of each incremental computations operator described is: described timer completes the timing timing statistics point to incremental computations time value corresponding to each incremental computations operator described, and described source operator is that in described streaming computing application model, data stream walks first operator upwards;

Send a notification message to described working node, described routing information is comprised in described notification message, according to described routing information, described notification message is delivered to each incremental computations operator described from described source operator to make described working node, and according to the part processing logic of each incremental computations operator described, incremental computations is carried out to the data of described business.

Third aspect present invention provides a kind of device of business processing, described application of installation is in stream calculation system, described stream calculation system also comprises main controlled node, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business, described device comprises:

Acquiring unit, for obtaining the operator of described main controlled node scheduling, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and the outlet operator of each iteration loop is configured with iterations corresponding to described each iteration loop, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop,

Receiving element, for receiving the data of described business;

Processing unit, the data of part processing logic to the business that described receiving element receives that the described each operator for obtaining according to described acquiring unit carries process;

Output unit, for at the part processing logic of described processing unit according to each outlet operator, when determining to have reached the iterations of the iteration loop at described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

In conjunction with the third aspect, in the implementation that the first is possible, described device also comprises: setting unit,

Described setting unit, in the data of described business that receive at described receiving element for described each iteration loop arranges the corresponding flag of iterations;

Described processing unit, specifically for for each iteration loop, when the data of described business carry out an iterative processing in described each iteration loop, the flag of the iterations that each iteration loop is corresponding described at the outlet operator place of described each iteration loop described setting unit being arranged adds 1;

Described output unit, time equal specifically for the iterations corresponding with each iteration loop described when the iterations indicated by the flag of iterations corresponding to each iteration loop described, export the data of the described business after each iteration loop iterative processing described from the outlet operator of each iteration loop described.

In conjunction with the first possible implementation of the third aspect, in the implementation that the second is possible,

Described processing unit, also for described output unit from after the outlet operator of each iteration loop described exports the data of the described business after each iteration loop iterative processing described, the iterations of the flag of iterations corresponding for each iteration loop described is made zero.

In conjunction with the implementation that the third aspect, the third aspect the first or the second are possible, in the implementation that the third is possible, described device also comprises transfer unit,

Described receiving element, also for the processing logic receiving notice message according to source operator, described source operator is that in described streaming computing application model, data stream walks first operator upwards, comprise the routing information arriving incremental computations operator from described source operator in described notification message, described incremental computations operator is the operator participating in incremental computations;

Described transfer unit, for the described routing information received according to described receiving element, described notification message is delivered to described incremental computations operator by described source operator, described notification message is that described main controlled node sends when arriving the triggered time of described incremental computations operator, and the triggered time of described incremental computations operator is: described timer completes the timing timing statistics point to incremental computations time value corresponding to described incremental computations operator;

Described processing unit, also for when described notification message is delivered to described incremental computations operator by described transfer unit, according to described participation incremental computations operator part processing logic, carries out incremental computations to the data of described business.

In conjunction with the third possible implementation of the third aspect, in the 4th kind of possible implementation,

Described processing unit, specifically for according to described participation incremental computations operator part processing logic, carry out data queue's switching, suspend the process of the data to the described business that the described triggered time receives, the data of the described business that priority processing received before the described triggered time.

Fourth aspect present invention provides a kind of device of business processing,

Described application of installation is in stream calculation system, and described stream calculation system also comprises working node, and described device comprises:

Acquiring unit, for obtaining the streaming computing application model of business, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and each iteration loop is configured with corresponding iterations, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop,

Dispensing unit, the iterations correspondence for the described each iteration loop obtained by described acquiring unit is configured to the outlet operator of described every iteration loop, and described outlet operator is that in described each iteration loop, data stream walks last operator upwards;

Scheduling unit, in described streaming computing application model after being configured by described dispensing unit, each Operator Scheduling is to described working node, processed by the data of part processing logic to described business carried according to described each operator to make described working node, according to the part processing logic of each outlet operator, when determining the iterations of the iteration loop having reached described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

In conjunction with fourth aspect, in the implementation that the first is possible, described device also comprises:

Determining unit, for determining incremental computations operator in the described streaming computing application model that described acquiring unit obtains, and the incremental computations time value of described incremental computations operator, described incremental computations operator is the operator participating in incremental computations;

Adding device, the incremental computations time value for the described incremental computations operator described determining unit determined is added in the time triggered queue of timer;

Described determining unit, also for for each incremental computations operator, when the triggered time of each incremental computations operator described in arriving described adding device and adding, determine the routing information arriving each incremental computations operator described from source operator, the triggered time of each incremental computations operator described is: described timer completes the timing timing statistics point to incremental computations time value corresponding to each incremental computations operator described, and described source operator is that in described streaming computing application model, data stream walks first operator upwards;

Transmitting element, for sending a notification message to described working node, the described routing information that described determining unit is determined is comprised in described notification message, according to described routing information, described notification message is delivered to each incremental computations operator described from described source operator to make described working node, and according to the part processing logic of each incremental computations operator described, incremental computations is carried out to the data of described business.

Fifth aspect present invention provides a kind of stream calculation system, and described stream calculation system comprises main controlled node and working node, and described system comprises:

Described main controlled node is used for:

The streaming computing application model of acquisition business, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and each iteration loop is configured with corresponding iterations, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop,

By each Operator Scheduling in the described streaming computing application model after configuration on described working node;

Described working node is used for:

Obtain the operator in the described streaming computing application model of described main controlled node scheduling;

Receive the data of described business;

In conjunction with the 5th aspect, in the implementation that the first is possible, described system also comprises:

Described main controlled node also for:

Determine incremental computations operator in described streaming computing application model, and the incremental computations time value of described incremental computations operator, described incremental computations operator is the operator participating in incremental computations;

Send a notification message to described working node, in described notification message, comprise described routing information;

Described working node also for:

Compared with prior art, the method of the business processing that the embodiment of the present invention provides, the streaming computing application module scheduling of iteration loop and iterations can will be comprised on working node by main controlled node, according to the part processing logic of operator in streaming computing application model, iterative processing is carried out to the data of business by working node, and the iterations of data in this iteration loop of this business is added up at the outlet operator place of each iteration loop, when the data of this business complete the iterative processing of an iteration loop, the data after iterative processing are exported from the outlet operator of this iteration loop.As can be seen here, the method for the business processing that the embodiment of the present invention provides, the data of business do not need first to store, but after receiving, directly iterative processing is carried out to the data of business, thus reduce the processing delay of the data of business, improve the real-time of data processing.

Accompanying drawing explanation

In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.

Fig. 1 is streaming computing application model schematic in the embodiment of the present invention;

Fig. 2 is an embodiment schematic diagram of the method for business processing in the embodiment of the present invention;

Fig. 3 is a schematic diagram of the streaming computing application model comprising iteration loop in the embodiment of the present invention;

Fig. 4 is another schematic diagram of the streaming computing application model comprising iteration loop in the embodiment of the present invention;

Fig. 5 is another embodiment schematic diagram of the method for business processing in the embodiment of the present invention;

Fig. 6 is a schematic diagram of the streaming computing application model comprising iteration loop and incremental computations in the embodiment of the present invention;

Fig. 7 is another embodiment schematic diagram of the method for business processing in the embodiment of the present invention;

Fig. 8 is another embodiment schematic diagram of the method for business processing in the embodiment of the present invention;

Fig. 9 is another embodiment schematic diagram of the method for business processing in the embodiment of the present invention;

Figure 10 is another embodiment schematic diagram of the method for business processing in the embodiment of the present invention;

Figure 11 is an embodiment schematic diagram of the device of business processing in the embodiment of the present invention;

Figure 12 is another embodiment schematic diagram of the device of business processing in the embodiment of the present invention;

Figure 13 is another embodiment schematic diagram of the device of business processing in the embodiment of the present invention;

Figure 14 is another embodiment schematic diagram of the device of business processing in the embodiment of the present invention;

Figure 15 is another embodiment schematic diagram of the device of business processing in the embodiment of the present invention;

Figure 16 is another embodiment schematic diagram of the device of business processing in the embodiment of the present invention;

Figure 17 is another embodiment schematic diagram of the device of business processing in the embodiment of the present invention;

Figure 18 is an embodiment schematic diagram of stream calculation system in the embodiment of the present invention.

Embodiment

The embodiment of the present invention provides a kind of method of business processing, and the data of the business received directly are carried out iterative processing, does not need first to store, can reduce the processing delay of the data of business, improves real-time.The embodiment of the present invention additionally provides corresponding Apparatus and system.Below be described in detail respectively.

The present invention program is understood better in order to make those skilled in the art person, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the embodiment of a part of the present invention, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, should belong to the scope of protection of the invention.

Stream calculation system in the embodiment of the present invention comprises: main controlled node and working node, when clustered deploy(ment), described main controlled node has one usually, but have main controlled node for subsequent use, working node can have one or more, main controlled node can be the physical node be separated with working node, and when unit is disposed, main controlled node and working node can be the master control logic unit be deployed on same physical node.Stream calculation system is used for dispatching and processing business, and detailed process is, main controlled node is responsible for traffic scheduling to the enterprising row relax of working node.A physical node can comprise a working node, also can comprise multiple working node, and the quantity of the working node that physical node comprises depends on the hardware physical resource of this physical node.A working node can be interpreted as it is a hardware physical resource.Belonging between the working node in same physical node adopts the mode of process communication to communicate, and belongs between the working node in different physical node and adopts the mode of network service to communicate.Stream calculation system can by traffic scheduling to the enterprising row relax of working node.Streaming computing application model can be set up in advance for the business operated in stream calculation system, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, and the operator in described streaming computing application model carries the computational logic for the treatment of the subservice corresponding with described operator in described business.Physical node in the embodiment of the present invention refers to physical equipment.

As shown in Figure 2, stream calculation system comprises main controlled node, working node 1, working node 2 and working node 3.Working node 1, working node 2 belong to physical node 1, and working node 3 belongs to physical node 2.

Streaming computing application module scheduling shown in Fig. 2 is carried out business processing by main controlled node to these three working nodes of working node 1, working node 2 and working node 3.Streaming computing application model shown in Fig. 2 is the logic relation picture of the data stream trend comprised between ten operators of P1 to P10 and this ten operators.Iterations corresponding for secondary iteration ring Loop2 to the outlet operator P8 of Loop1, configures to the outlet operator P7 of Loop2 for 10 times by main controlled node by iterations 100 configurations corresponding for the first iteration loop Loop1.

For P7 and P8 configures after iterations, main controlled node by streaming computing application module scheduling on working node 1, working node 2 and working node 3.Concrete scheduling rule is not the emphasis of the embodiment of the present invention, and too much introduction is not done at this place, is understandable that: main controlled node can according to each operator on the hardware resource situation scheduling streaming computing application model of each working node.

Be scheduled for operator P1, P2 and P3 on working node 1 and be scheduled for being communicated by process mode between operator P4, P5 and P6 on working node 2, P1, P2 and P3, and P4, P5 and P6 and be scheduled for working node 3 passes through network service between operator P7, P8, P9 and P10.

Streaming computing application model in the embodiment of the present invention comprises directed cyclic graph (Directed Cyclic Graph, DGG).

In stream calculation system, the course of work of main controlled node and working node can be understood as:

Main controlled node obtains the streaming computing application model of described business, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and each iteration loop is configured with corresponding iterations, the iterations of described each iteration loop correspondence is configured on the outlet operator of described every iteration loop, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, by each Operator Scheduling in the described streaming computing application model after configuration on described working node, described working node is processed by the data of part processing logic to described business carried according to described each operator, according to the part processing logic of each outlet operator, when determining the iterations of the iteration loop having reached described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

Described streaming computing application model is that user or developer set up in advance according to described business, streaming computing application model is for representing the logical model that streaming is applied, streaming computing application model can be consulted Fig. 3 and understand, streaming computing application model as shown in Figure 3 comprises 10 operator P1-P10, wherein the direction of arrow is the trend of data stream, P1 is first operator that data stream walks upwards, P1 can be claimed to be source operator, two iteration loop are had in streaming computing application model shown in Fig. 3, be respectively first iteration loop Loop1, need iteration 100 times, second iteration loop Loop2, need iteration 10 times.The iterative process of data in this streaming computing application model of business can be: the data of business are from the operator P1 of source, enter the entrance operator P2 of first iteration loop Loop1, then the first time interative computation in Loop1 is carried out at operator P3 place, enter the entrance operator P4 of second iteration loop Loop2 again, then the first time interative computation in Loop2 is carried out at operator P5 and operator P2 place, determine only to have carried out an interative computation in Loop2 at the outlet operator place of Loop2, also do not arrive 10 times, then the data of business are turned back to P5 according to the second time interative computation in Loop2, repeat this process, until P7 determines the data of business in Loop2 according to 10 interative computations, then P7 outputs to P8 by Loop2 according to the data of the business after 10 iteration, P8 determines the data of business in Loop1 only according to 1 interative computation, there is no arrival 100 times, then the data of business are turned back to P3, again according to the process of P3 to P8, until P8 determines the data of business in Loop1 according to 100 interative computations.The data of business are exported to P9 by P8, and the data of business are transferred to P10 by P9, complete whole iterative process.

As shown in Figure 4, comprise source operator In in streaming computing application model shown in Fig. 4, terminate operator Out, two iteration loop are had in streaming computing application model shown in Fig. 4, be respectively first iteration loop Loop1, need iteration 2 times, second iteration loop Loop2, need iteration 2 times, comprise Loop2 in Loop1, in Loop2, comprise the entrance operator I of Loop2 ₂, iteration operator B and outlet operator E ₂, it is I that Loop1 is incorporated into mental arithmetic ₁, outlet operator is E ₁, the iteration operator A of Loop1.

After main controlled node gets the streaming computing application model shown in Fig. 4, iterations 2 corresponding for Loop1 is configured to E ₁iterations 2 corresponding for Loop2 is configured to E2, then by the streaming computing application module scheduling shown in Fig. 4 to working node, like this, working node just the processing logic of streaming computing application model according to Fig. 4 can carry out iterative processing to the data of business.

Fig. 5, for after the operator in the streaming computing application model shown in Fig. 4 is scheduled for working node, according to processing procedure schematic diagram during business processing, contains 10 width subgraphs in business procession, marks respectively with 1-10 in Fig. 5.

In Fig. 5, label is in the subgraph of 1, after source operator In receives the data of business, in the data of business, the first flag and the second flag are set for two iteration loop, first flag represents the iterations of the data of business in Loop1, the second flag represents the iterations in Loop2, and the initial value of source operator In place first flag and the second flag is all 0.Be in the subgraph of 2 at label, the data of business enter the entrance operator I of Loop1 ₁, the first flag is revised as 1.Be in the subgraph of 3 at label, the data of business enter the entrance operator I of Loop2 ₂, the second flag is revised as 1.Be in the subgraph of 4 at label, E ₂the iterations determined in Loop2 from the second flag does not reach 2 times, then the data of business are returned to operator B, and according to the second time iterative processing in Loop2, the second flag is revised as 2.Be in the subgraph of 5 at label, E ₂the iterations determined in Loop2 from the second flag has reached 2 times, then the data of business are outputted to E ₁.Be in the subgraph of 6 at label, E ₁determine that iterations does not reach 2 times from the first flag, then the data of business are turned back to operator A, according to the second time iterative processing in Loop1, the first flag is revised as 2.Be in the subgraph of 7 at label, operator I ₂be revised as 0 by 2 of secondary iteration flag, when the data of business are again according to iterative process in Loop2, the second flag be revised as 1 again.Label for 8 with institute in the subgraph of 9 according to process and label be 4 identical with the process of 5, do not do at this and too much repeat.Be E in the subgraph of 10 at label ₂the iterations determined in Loop1 from the first flag has reached 2 times, then the data of business are outputted to operator Out, terminates whole iterative process.

In the embodiment of the present invention, can also carry out incremental computations in the process of carrying out iterative processing, user only needs the time arranging incremental computations on streaming computing application model.As shown in Figure 6, operator A is incremental computations operator, incremental computations operator is exactly the operator participating in incremental computations, the incremental computations time value of operator A is 3min, after main controlled node obtains the streaming computing application model shown in Fig. 6, the incremental computations time value 3min of operator A can also be obtained, then the incremental computations time value 3min of operator A is added in the time triggered queue of timer.After main controlled node is by the streaming computing application module scheduling to working node shown in Fig. 6, consult Fig. 7, timer is that operator A carries out timing, if every 3min needs to carry out an incremental computations, then timer can be set to 3min, after the 3min timing of timer terminates, operator notification unit obtains the routing information arriving operator A from source operator In, routing information can represent by the form of operator name+outbound port number, as: the routing information arriving operator A can be source operator In0, represent and can arrive operator A from No. 0 outbound port of source operator In, then operator notification unit builds notification message, In0 is comprised in this notification message, after source operator In receives notification message, notification message is passed to operator A, after operator A receives notification message, suspend the process to the data newly receiving business, priority processing receive notification message before data, data before this ensure that notification message can complete incremental computations before at the appointed time putting, thus ensure that the accuracy of incremental computations time.Not middle operator is experienced from source operator to operator A in the embodiment of the present invention, in fact one or more middle operator can be had in source operator and operator A, notification message can be passed to operator A by middle operator by source operator, each middle operator upon receiving the notification message, all can priority processing receive notification message before the data of business.

As shown in Figure 8, Fig. 8 is schematic diagram in incremental computations process, comprises 4 width subgraphs, carry out label with 1 to 4 in Fig. 8, and wherein in the subgraph shown in label 1, have two queues in operator A, queue 1 and queue 2, operator A receives new data, then leave in queue 1.Label is in the subgraph of 2, and source operator In receives notification message notify, and notify is transferred to operator A by source operator In.Label is in the subgraph of 3, after operator A receives notification message, switches receiving queue, the data of the business in operator A priority processing queue 2.Label is in the subgraph of 4, after the business processing in queue 2 is complete, and the data of the business in reprocessing queue 1.

Working node 30 in the embodiment of the present invention can be understood as working node cluster, in fact can be the flow process that multiple working node completes business processing in the embodiment of the present invention jointly.

Consult Fig. 9, the method method of a kind of business processing that the embodiment of the present invention provides is applied to the working node in stream calculation system, described stream calculation system also comprises main controlled node, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business, described method comprises:

101, described working node obtains the operator of described main controlled node scheduling, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and the outlet operator of each iteration loop is configured with iterations corresponding to described each iteration loop, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop.

102, the data of described business are received.

103, the data of part processing logic to described business carried according to described each operator process, at the part processing logic according to each outlet operator, when determining to have reached the iterations of the iteration loop at described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

The method of the business processing that the embodiment of the present invention provides, described method is applied to the working node in stream calculation system, described stream calculation system also comprises main controlled node, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business, described method comprises: described working node obtains the operator of described main controlled node scheduling, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and the outlet operator of each iteration loop is configured with iterations corresponding to described each iteration loop, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, receive the data of described business, process according to the data of part processing logic to described business that described each operator carries, at the part processing logic according to each outlet operator, when determining to have reached the iterations of the iteration loop at described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.Compared with prior art, the method of the business processing that the embodiment of the present invention provides, the streaming computing application module scheduling of iteration loop and iterations can will be comprised on working node by main controlled node, according to the part processing logic of operator in streaming computing application model, iterative processing is carried out to the data of business by working node, and the iterations of data in this iteration loop of this business is added up at the outlet operator place of each iteration loop, when the data of this business complete the iterative processing of an iteration loop, the data after iterative processing are exported from the outlet operator of this iteration loop.As can be seen here, the method for the business processing that the embodiment of the present invention provides, the data of business do not need first to store, but after receiving, directly iterative processing is carried out to the data of business, thus reduce the processing delay of the data of business, improve the real-time of data processing.

Alternatively, on the basis of embodiment corresponding to above-mentioned Fig. 9, in first embodiment of the method for the business processing that the embodiment of the present invention provides, after the data of the described business of described reception, described method also comprises:

The described data of part processing logic to described business carried according to described each operator process, at the part processing logic according to each outlet operator, when determining to have reached the iterations of the iteration loop at described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator, specifically can comprise:

Alternatively, on the basis of first embodiment of the method for above-mentioned business processing, in second embodiment of the method for the business processing that the embodiment of the present invention provides, after the described outlet operator from each iteration loop described exports the data of the described business after each iteration loop iterative processing described, described method can also comprise:

Alternatively, on the basis of the embodiment of the method for above-mentioned business processing, first or second embodiment, in the 3rd embodiment of the method for the business processing that the embodiment of the present invention provides, described method can also comprise:

Alternatively, on the basis of the 3rd embodiment of the method for above-mentioned business processing, in 4th embodiment of the method for the business processing that the embodiment of the present invention provides, described according to described participation incremental computations operator part processing logic, incremental computations is carried out to the data of described business, specifically can comprise:

The embodiment that above Fig. 9 is corresponding and arbitrary embodiment, the description can consulting the procedure division of the business processing performed by Fig. 2-Fig. 8 some work node is understood, and this place does not do and too much repeats.

Consult Figure 10, the method for the business processing that the embodiment of the present invention provides is applied to the main controlled node in stream calculation system, and described stream calculation system also comprises working node, and described method comprises:

201, described main controlled node obtains the streaming computing application model of business, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and each iteration loop is configured with corresponding iterations, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop.

202, be configured on the outlet operator of described every iteration loop by the iterations of described each iteration loop correspondence, described outlet operator is that in described each iteration loop, data stream walks last operator upwards.

203, by configuration after described streaming computing application model in each Operator Scheduling on described working node, processed by the data of part processing logic to described business carried according to described each operator to make described working node, according to the part processing logic of each outlet operator, when determining the iterations of the iteration loop having reached described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

The method of the business processing that the embodiment of the present invention provides, described method is applied to the main controlled node in stream calculation system, described stream calculation system also comprises working node, described method comprises: described main controlled node obtains the streaming computing application model of business, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and each iteration loop is configured with corresponding iterations, be configured on the outlet operator of described every iteration loop by the iterations of described each iteration loop correspondence, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, by each Operator Scheduling in the described streaming computing application model after configuration on described working node, processed by the data of part processing logic to described business carried according to described each operator to make described working node, according to the part processing logic of each outlet operator, when determining the iterations of the iteration loop having reached described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.Compared with prior art, the method of the business processing that the embodiment of the present invention provides, the streaming computing application module scheduling of iteration loop and iterations can will be comprised on working node by main controlled node, according to the part processing logic of operator in streaming computing application model, iterative processing is carried out to the data of business by working node, and the iterations of data in this iteration loop of this business is added up at the outlet operator place of each iteration loop, when the data of this business complete the iterative processing of an iteration loop, the data after iterative processing are exported from the outlet operator of this iteration loop.As can be seen here, the method for the business processing that the embodiment of the present invention provides, the data of business do not need first to store, but after receiving, directly iterative processing is carried out to the data of business, thus reduce the processing delay of the data of business, improve the real-time of data processing.

Alternatively, on the basis of embodiment corresponding to above-mentioned Figure 10, in another embodiment of the method for the business processing that the embodiment of the present invention provides, described method also comprises:

The embodiment that above Figure 10 is corresponding and embodiment, the description can consulting the procedure division of the business processing performed by Fig. 2-Fig. 8 part main controlled node is understood, and this place does not do and too much repeats.

Consult Figure 11, the device of the business processing that the embodiment of the present invention provides, described application of installation is in stream calculation system, described stream calculation system also comprises main controlled node, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business, described device 30 comprises:

Acquiring unit 301, for obtaining the operator of described main controlled node scheduling, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and the outlet operator of each iteration loop is configured with iterations corresponding to described each iteration loop, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop,

Receiving element 302, for receiving the data of described business;

Processing unit 303, the data of part processing logic to the business that described receiving element 302 receives that the described each operator for obtaining according to described acquiring unit 301 carries process;

Output unit 304, for at the part processing logic of described processing unit 303 according to each outlet operator, when determining to have reached the iterations of the iteration loop at described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

The device of the business processing that the embodiment of the present invention provides, described application of installation is in stream calculation system, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business, described stream calculation system also comprises main controlled node, described device comprises: acquiring unit, for obtaining the operator in the streaming computing application model of described main controlled node scheduling, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and the outlet operator of each iteration loop is configured with iterations corresponding to described each iteration loop, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, receiving element, for receiving the data of described business, processing unit, the data of part processing logic to the business that described receiving element receives that the described each operator for obtaining according to described acquiring unit carries process, output unit, for at the part processing logic of described processing unit according to each outlet operator, when determining to have reached the iterations of the iteration loop at described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.Compared with prior art, the device of the business processing that the embodiment of the present invention provides, according to the part processing logic of operator in streaming computing application model, iterative processing is carried out to the data of business, and the iterations of data in this iteration loop of this business is added up at the outlet operator place of each iteration loop, when the data of this business complete the iterative processing of an iteration loop, export the data after iterative processing from the outlet operator of this iteration loop.As can be seen here, the device of the business processing that the embodiment of the present invention provides, the data of business do not need first to store, but after receiving, directly iterative processing is carried out to the data of business, thus reduce the processing delay of the data of business, improve the real-time of data processing.

Alternatively, on the basis of embodiment corresponding to above-mentioned Figure 11, consult Figure 12, in first embodiment of the device 30 of the business processing that the embodiment of the present invention provides, described device also comprises: setting unit,

Described setting unit 305, in the data of described business that receive at described receiving element 302 for described each iteration loop arranges the corresponding flag of iterations;

Described processing unit 303, specifically for for each iteration loop, when the data of described business carry out an iterative processing in described each iteration loop, the flag of the iterations that each iteration loop is corresponding described at the outlet operator place of described each iteration loop described setting unit 305 being arranged adds 1;

Described output unit 304, time equal specifically for the iterations corresponding with each iteration loop described when the iterations indicated by the flag of iterations corresponding to each iteration loop described, export the data of the described business after each iteration loop iterative processing described from the outlet operator of each iteration loop described.

Alternatively, on the basis of embodiment corresponding to above-mentioned Figure 12, in second embodiment of the device 30 of the business processing that the embodiment of the present invention provides,

Described processing unit 303, also for described output unit from after the outlet operator of each iteration loop described exports the data of the described business after each iteration loop iterative processing described, the iterations of the flag of iterations corresponding for each iteration loop described is made zero.

Alternatively, on the basis of embodiment corresponding to above-mentioned Figure 11, consult Figure 13, in the 3rd embodiment of the device 30 of the business processing that the embodiment of the present invention provides, described device 30 also comprises transfer unit 306,

Described receiving element 302, also for the processing logic receiving notice message according to source operator, described source operator is that in described streaming computing application model, data stream walks first operator upwards, comprise the routing information arriving incremental computations operator from described source operator in described notification message, described incremental computations operator is the operator participating in incremental computations;

Described transfer unit 306, for the described routing information received according to described receiving element 302, described notification message is delivered to described incremental computations operator by described source operator, described notification message is that described main controlled node sends when arriving the triggered time of described incremental computations operator, and the triggered time of described incremental computations operator is: described timer completes the timing timing statistics point to incremental computations time value corresponding to described incremental computations operator;

Described processing unit 303, also for when described notification message is delivered to described incremental computations operator by described transfer unit 306, according to described participation incremental computations operator part processing logic, carries out incremental computations to the data of described business.

Alternatively, on the basis of embodiment corresponding to above-mentioned Figure 13, in the 4th embodiment of the device 30 of the business processing that the embodiment of the present invention provides,

Described processing unit 303, specifically for according to described participation incremental computations operator part processing logic, carry out data queue's switching, suspend the process of the data to the described business that the described triggered time receives, the data of the described business that priority processing received before the described triggered time.

The embodiment of the device of above business processing or embodiment, the description can consulting the procedure division of the business processing performed by Fig. 2-Fig. 8 some work node is understood, and this place does not do and too much repeats.

Consult Figure 14, the device of the business processing that the embodiment of the present invention provides, described application of installation is in stream calculation system, and described stream calculation system also comprises working node, and described device 40 comprises:

Acquiring unit 401, for obtaining the streaming computing application model of business, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and each iteration loop is configured with corresponding iterations;

Dispensing unit 402, the iterations correspondence for the described each iteration loop obtained by described acquiring unit 401 is configured to the outlet operator of described every iteration loop, and described outlet operator is that in described each iteration loop, data stream walks last operator upwards;

Scheduling unit 403, in described streaming computing application model after being configured by described dispensing unit 402, each Operator Scheduling is to described working node, processed by the data of part processing logic to described business carried according to described each operator to make described working node, according to the part processing logic of each outlet operator, when determining the iterations of the iteration loop having reached described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.

The application of installation of the business processing provided in the embodiment of the present invention is in stream calculation system, described stream calculation system also comprises working node, described device comprises: acquiring unit 401 obtains the streaming computing application model of business, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and each iteration loop is configured with corresponding iterations, the iterations correspondence of described each iteration loop that described acquiring unit 401 obtains by dispensing unit 402 is configured on the outlet operator of described every iteration loop, and described outlet operator is that in described each iteration loop, data stream walks last operator upwards, in described streaming computing application model after described dispensing unit 402 configures by scheduling unit 403, each Operator Scheduling is on described working node, processed by the data of part processing logic to described business carried according to described each operator to make described working node, according to the part processing logic of each outlet operator, when determining the iterations of the iteration loop having reached described each outlet operator place to the iterations of the data of described business, then the data of described business are exported from described each outlet operator.Compared with prior art, the device of the business processing that the embodiment of the present invention provides, streaming computing application model can be dispatched, thus ensure that working node can carry out iterative processing according to the part processing logic of operator in streaming computing application model to the data of business, and the iterations of data in this iteration loop of this business is added up at the outlet operator place of each iteration loop, when the data of this business complete the iterative processing of an iteration loop, export the data after iterative processing from the outlet operator of this iteration loop.As can be seen here, the device of the business processing that the embodiment of the present invention provides, can ensure that the data of business in working node do not need first to store, but after receiving, directly iterative processing is carried out to the data of business, thus reduce the processing delay of the data of business, improve the real-time of data processing.

Alternatively, on the basis of embodiment corresponding to above-mentioned Figure 14, consult Figure 15, in another embodiment of the device of the business processing that the embodiment of the present invention provides, described device 40 also comprises:

Determining unit 404, for determining incremental computations operator in the described streaming computing application model that described acquiring unit 401 obtains, and the incremental computations time value of described incremental computations operator, described incremental computations operator is the operator participating in incremental computations;

Adding device 405, the incremental computations time value for the described incremental computations operator described determining unit 404 determined is added in the time triggered queue of timer;

Described determining unit 404, also for for each incremental computations operator, when the triggered time of each incremental computations operator described in arriving described adding device 405 and adding, determine the routing information arriving each incremental computations operator described from source operator, the triggered time of each incremental computations operator described is: described timer completes the timing timing statistics point to incremental computations time value corresponding to each incremental computations operator described, and described source operator is that in described streaming computing application model, data stream walks first operator upwards;

Transmitting element 406, for sending a notification message to described working node, the described routing information that described determining unit 404 is determined is comprised in described notification message, according to described routing information, described notification message is delivered to each incremental computations operator described from described source operator to make described working node, and according to the part processing logic of each incremental computations operator described, incremental computations is carried out to the data of described business.

The embodiment of the device of above business processing or embodiment, the description can consulting the procedure division of the business processing performed by Fig. 2-Fig. 8 part main controlled node is understood, and this place does not do and too much repeats.

Figure 16 is the structural representation of the device 30 of embodiment of the present invention business processing.The device 30 of business processing can comprise input equipment 310, output device 320, processor 330 and storer 340.

The device 30 of the business processing that the embodiment of the present invention provides is applied to stream calculation system, described stream calculation system also comprises main controlled node, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business.

Storer 340 can comprise ROM (read-only memory) and random access memory, and provides instruction and data to processor 330.A part for storer 340 can also comprise nonvolatile RAM (NVRAM).

Storer 340 stores following element, executable module or data structure, or their subset, or their superset:

Operational order: comprise various operational order, for realizing various operation.

Operating system: comprise various system program, for realizing various basic business and processing hardware based task.

In embodiments of the present invention, the operational order (this operational order can store in an operating system) that processor 330 stores by calling storer 340, performs and operates as follows:

Obtain the operator in the streaming computing application model of described main controlled node scheduling, described streaming computing application model is the logic relation picture comprising data stream trend between operator and operator set up in advance for described business, each operator is for carrying the part processing logic in processing logic corresponding to described business, operator in described streaming computing application model and the data stream between described operator are moved towards to form at least one iteration loop, and the outlet operator of each iteration loop is configured with iterations corresponding to described each iteration loop, described outlet operator is that in described each iteration loop, data stream walks last operator upwards, iterations corresponding to described each iteration loop represents that the data of described business need to be iterated the number of times of process in described each iteration loop,

The data of described business are received by input equipment 310;

The device 30 of the business processing that the embodiment of the present invention provides, according to the part processing logic of operator in streaming computing application model, iterative processing is carried out to the data of business, and the iterations of data in this iteration loop of this business is added up at the outlet operator place of each iteration loop, when the data of this business complete the iterative processing of an iteration loop, export the data after iterative processing from the outlet operator of this iteration loop.As can be seen here, the device of the business processing that the embodiment of the present invention provides, the data of business do not need first to store, but after receiving, directly iterative processing is carried out to the data of business, thus reduce the processing delay of the data of business, improve the real-time of data processing.

The operation of the device 30 of processor 330 service control process, processor 330 can also be called CPU (Central Processing Unit, CPU (central processing unit)).Storer 340 can comprise ROM (read-only memory) and random access memory, and provides instruction and data to processor 330.A part for storer 340 can also comprise nonvolatile RAM (NVRAM).In concrete application, each assembly of the device 30 of business processing is coupled by bus system 350, and wherein bus system 350 is except comprising data bus, can also comprise power bus, control bus and status signal bus in addition etc.But for the purpose of clearly demonstrating, in the drawings various bus is all designated as bus system 350.

The method that the invention described above embodiment discloses can be applied in processor 330, or is realized by processor 330.Processor 330 may be a kind of integrated circuit (IC) chip, has the processing power of signal.In implementation procedure, each step of said method can be completed by the instruction of the integrated logic circuit of the hardware in processor 330 or software form.Above-mentioned processor 330 can be general processor, digital signal processor (DSP), special IC (ASIC), ready-made programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components.Can realize or perform disclosed each method, step and the logic diagram in the embodiment of the present invention.The processor etc. of general processor can be microprocessor or this processor also can be any routine.Step in conjunction with the method disclosed in the embodiment of the present invention directly can be presented as that hardware decoding processor is complete, or combines complete by the hardware in decoding processor and software module.Software module can be positioned at random access memory, flash memory, ROM (read-only memory), in the storage medium of this area maturations such as programmable read only memory or electrically erasable programmable storer, register.This storage medium is positioned at storer 340, and processor 330 reads the information in storer 340, completes the step of said method in conjunction with its hardware.

Alternatively, described processor 330 also in the data of described business for described each iteration loop arranges the corresponding flag of iterations;

Described processor 330 is specifically for for each iteration loop, when the data of described business carry out an iterative processing in described each iteration loop, the flag of the outlet operator place of the described each iteration loop iterations corresponding to each iteration loop described adds 1;

When described output device 320 is equal specifically for the iterations corresponding with each iteration loop described when the iterations indicated by the flag of iterations corresponding to each iteration loop described, export the data of the described business after each iteration loop iterative processing described from the outlet operator of each iteration loop described.

Alternatively, the iterations of the flag of iterations corresponding for each iteration loop described also for after the data from the described business after outlet operator output each iteration loop iterative processing described of each iteration loop described, makes zero by described processor 330.

Alternatively, described input equipment 410 is also for the processing logic receiving notice message according to source operator, described source operator is that in described streaming computing application model, data stream walks first operator upwards, comprise the routing information arriving incremental computations operator from described source operator in described notification message, described incremental computations operator is the operator participating in incremental computations;

Described processor 330 is also for according to described routing information, described notification message is delivered to described incremental computations operator by described source operator, described notification message is that described main controlled node sends when arriving the triggered time of described incremental computations operator, the triggered time of described incremental computations operator is: described timer completes the timing timing statistics point to incremental computations time value corresponding to described incremental computations operator, according to described participation incremental computations operator part processing logic, incremental computations is carried out to the data of described business.

Described processor 330 is specifically for according to described participation incremental computations operator part processing logic, carry out data queue's switching, suspend the process to the data of the described business that the described triggered time receives, the data of the described business that priority processing received before the described triggered time.

Figure 17 is the structural representation of the device 40 of embodiment of the present invention business processing.The device 40 of business processing can comprise input equipment 410, output device 420, processor 430 and storer 440.

The device 40 of the business processing that the embodiment of the present invention provides is applied to stream calculation system, and described stream calculation system also comprises working node.

Storer 440 can comprise ROM (read-only memory) and random access memory, and provides instruction and data to processor 430.A part for storer 440 can also comprise nonvolatile RAM (NVRAM).

Storer 440 stores following element, executable module or data structure, or their subset, or their superset:

In embodiments of the present invention, the operational order (this operational order can store in an operating system) that processor 430 stores by calling storer 440, performs and operates as follows:

The device 40 of the business processing that the embodiment of the present invention provides, streaming computing application model can be dispatched, thus ensure that working node can carry out iterative processing according to the part processing logic of operator in streaming computing application model to the data of business, and the iterations of data in this iteration loop of this business is added up at the outlet operator place of each iteration loop, when the data of this business complete the iterative processing of an iteration loop, export the data after iterative processing from the outlet operator of this iteration loop.As can be seen here, the device of the business processing that the embodiment of the present invention provides, can ensure that the data of business in working node do not need first to store, but after receiving, directly iterative processing is carried out to the data of business, thus reduce the processing delay of the data of business, improve the real-time of data processing.

The operation of the device 40 of processor 430 service control process, processor 430 can also be called CPU (Central Processing Unit, CPU (central processing unit)).Storer 440 can comprise ROM (read-only memory) and random access memory, and provides instruction and data to processor 430.A part for storer 440 can also comprise nonvolatile RAM (NVRAM).In concrete application, each assembly of the device 40 of business processing is coupled by bus system 450, and wherein bus system 450 is except comprising data bus, can also comprise power bus, control bus and status signal bus in addition etc.But for the purpose of clearly demonstrating, in the drawings various bus is all designated as bus system 450.

The method that the invention described above embodiment discloses can be applied in processor 430, or is realized by processor 430.Processor 430 may be a kind of integrated circuit (IC) chip, has the processing power of signal.In implementation procedure, each step of said method can be completed by the instruction of the integrated logic circuit of the hardware in processor 430 or software form.Above-mentioned processor 430 can be general processor, digital signal processor (DSP), special IC (ASIC), ready-made programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components.Can realize or perform disclosed each method, step and the logic diagram in the embodiment of the present invention.The processor etc. of general processor can be microprocessor or this processor also can be any routine.Step in conjunction with the method disclosed in the embodiment of the present invention directly can be presented as that hardware decoding processor is complete, or combines complete by the hardware in decoding processor and software module.Software module can be positioned at random access memory, flash memory, ROM (read-only memory), in the storage medium of this area maturations such as programmable read only memory or electrically erasable programmable storer, register.This storage medium is positioned at storer 440, and processor 430 reads the information in storer 440, completes the step of said method in conjunction with its hardware.

Alternatively, described processor 430 is also for determining incremental computations operator in described streaming computing application model, and the incremental computations time value of described incremental computations operator, and described incremental computations operator is the operator participating in incremental computations; The incremental computations time value of described incremental computations operator is added in the time triggered queue of timer; For each incremental computations operator, when arriving the triggered time of each incremental computations operator described, determine the routing information arriving each incremental computations operator described from source operator, the triggered time of each incremental computations operator described is: described timer completes the timing timing statistics point to incremental computations time value corresponding to each incremental computations operator described, and described source operator is that in described streaming computing application model, data stream walks first operator upwards;

Described output device 420 sends a notification message to described working node, described routing information is comprised in described notification message, according to described routing information, described notification message is delivered to each incremental computations operator described from described source operator to make described working node, and according to the part processing logic of each incremental computations operator described, incremental computations is carried out to the data of described business.

Consult Figure 18, an embodiment of the stream calculation system that the embodiment of the present invention provides comprises: described stream calculation system comprises main controlled node 40 and working node 30, it is characterized in that, described system comprises:

Described main controlled node 40 for:

Described working node 30 for:

Receive the data of described business;

Compared with prior art, the stream calculation system that the embodiment of the present invention provides, the streaming computing application module scheduling of iteration loop and iterations can will be comprised on working node by main controlled node, according to the part processing logic of operator in streaming computing application model, iterative processing is carried out to the data of business by working node, and the iterations of data in this iteration loop of this business is added up at the outlet operator place of each iteration loop, when the data of this business complete the iterative processing of an iteration loop, export the data after iterative processing from the outlet operator of this iteration loop.As can be seen here, the method for the business processing that the embodiment of the present invention provides, the data of business do not need first to store, but after receiving, directly iterative processing is carried out to the data of business, thus reduce the processing delay of the data of business, improve the real-time of data processing.

Alternatively, in another embodiment of the stream calculation system that the embodiment of the present invention provides,

Described main controlled node 40 also for:

Described working node 30 also for:

The stream calculation system that the embodiment of the present invention provides, can also ensure the accuracy of incremental computations time.

One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is that the hardware (such as processor) that can carry out instruction relevant by program has come, this program can be stored in a computer-readable recording medium, and storage medium can comprise: ROM, RAM, disk or CD etc.

Above the method for the business processing that the embodiment of the present invention provides, device and system are described in detail, apply specific case herein to set forth principle of the present invention and embodiment, the explanation of above embodiment just understands method of the present invention and core concept thereof for helping; Meanwhile, for one of ordinary skill in the art, according to thought of the present invention, all will change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.

Claims

1. the method for a business processing, it is characterized in that, described method is applied to the working node in stream calculation system, described stream calculation system also comprises main controlled node, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business, described method comprises:

Receive the data of described business;

2. method according to claim 1, is characterized in that, after the data of the described business of described reception, described method also comprises:

3. method according to claim 2, is characterized in that, after the described outlet operator from each iteration loop described exports the data of the described business after each iteration loop iterative processing described, described method also comprises:

4., according to the arbitrary described method of claim 1-3, it is characterized in that, described method also comprises:

5. method according to claim 4, is characterized in that, described according to described participation incremental computations operator part processing logic, carries out incremental computations, specifically comprise the data of described business:

6. a method for business processing, is characterized in that, described method is applied to the main controlled node in stream calculation system, and described stream calculation system also comprises working node, and described method comprises:

7. method according to claim 6, is characterized in that, described method also comprises:

8. the device of a business processing, it is characterized in that, described application of installation is in stream calculation system, described stream calculation system also comprises main controlled node, described main controlled node according to the one or more operator of streaming computing application module scheduling to described working node, to process the data of business, described device comprises:

Receiving element, for receiving the data of described business;

9. device according to claim 8, is characterized in that, described device also comprises: setting unit,

10. device according to claim 9, is characterized in that,

11.-10 arbitrary described devices according to Claim 8, it is characterized in that, described device also comprises transfer unit,

12. devices according to claim 11, is characterized in that,

The device of 13. 1 kinds of business processing, is characterized in that, described application of installation is in stream calculation system, and described stream calculation system also comprises working node, and described device comprises:

14. devices according to claim 13, is characterized in that, described device also comprises:

15. 1 kinds of stream calculation systems, described stream calculation system comprises main controlled node and working node, it is characterized in that, described system comprises:

Described main controlled node is used for:

Described working node is used for:

Receive the data of described business;

16. systems according to claim 15, is characterized in that, described system also comprises:

Described main controlled node also for:

Described working node also for: