CN102904919B - Stream processing method and distributed system for realizing stream processing - Google Patents

Stream processing method and distributed system for realizing stream processing Download PDF

Info

Publication number
CN102904919B
CN102904919B CN201110214487.XA CN201110214487A CN102904919B CN 102904919 B CN102904919 B CN 102904919B CN 201110214487 A CN201110214487 A CN 201110214487A CN 102904919 B CN102904919 B CN 102904919B
Authority
CN
China
Prior art keywords
data block
computing node
distributor
calculation
computational logic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201110214487.XA
Other languages
Chinese (zh)
Other versions
CN102904919A (en
Inventor
藤启明
李影
刘之育
王海川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to CN201110214487.XA priority Critical patent/CN102904919B/en
Publication of CN102904919A publication Critical patent/CN102904919A/en
Application granted granted Critical
Publication of CN102904919B publication Critical patent/CN102904919B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Computer And Data Communications (AREA)

Abstract

The invention relates to a stream processing method and a distributed system for realizing stream processing. The invention discloses a stream processing method for the distributed system. The distributed system comprises a distributor and a plurality of calculation nodes. The method comprises the following steps: in response to a received data block distributed by the distributor, each calculation node calls a corresponding calculation logic to calculate the data block and generates a calculation result; and the calculation node only transmits the part of the calculation result related to the calculation of the next data block to a calculation node in which the next data block is positioned.

Description

The distributed system of method for stream processing and the process of realization stream
Technical field
The present invention relates to stream process, particularly for the method for stream processing of distributed system and the distributed system of realization stream process.
Background technology
Stream treatment technology is widely used in the real time processing system in various field, process logically and/or the sequence of the event the recurred composition of time upper association, such as share certificate trade center, network monitoring, Web application, communication data management, the common feature of this kind of application is: real-time property is strong, data volume is very big, have quite high sudden, recur and constantly change.Stream treatment technology need Real-Time Monitoring continuous print data flow, in the process that data constantly change, carry out data analysis in real time, capture may be useful to user information, emergency is responded fast, processes in real time.
At present, stream treatment technology mainly adopts distributed account form, multiple computing node has allocated computational logic in advance, data flow is divided into multiple data block, according to the logical calculated of order complete paired data block on multiple computing node of computational logic, the process of usual next data block needs the result depending on previous data block, therefore need to carry out a large amount of transfer of data between computing node, statistics is had to show, data flow transmits 500K-1M message each second, and the about 1KB of each message, therefore 1GB is transmitted each second, and the transmission rate of data flow is also with the speed increment of every 3 years 40%.Therefore, when the network bandwidth is limited, between computing node, frequently carry out transfer of data inevitably cause network congestion, order execution delay.
Therefore need a kind of efficiently, save the method for stream processing of network bandwidth resources.
Summary of the invention
Based on the problems referred to above, the invention provides a kind of efficient method for stream processing and realize flowing the distributed system processed.
According to a first aspect of the invention, a kind of method for stream processing for distributed system is provided, this distributed system comprises distributor and multiple computing node, the method comprises: in response to the data block receiving distributor distribution, and described computing node calls corresponding computational logic and calculates described data block and produce result of calculation; And described computing node only calculates to next data block the computing node that the described result of calculation of relevant part sends next data block place to.
According to a second aspect of the invention, provide a kind of distributed system of carrying out flowing process, this distributed system comprises: distributor; Multiple computing node, is configured to the data block in response to receiving distributor distribution, calls corresponding computational logic and calculates described data block and produce result of calculation; And only calculate to next data block the computing node that the described result of calculation of relevant part sends next data block place to.
Method for stream processing and distributed system according to an embodiment of the invention, reduces the mass data carrying out transmitting between computing node, saves network bandwidth resources.
Accompanying drawing explanation
By reference to the accompanying drawings, by reference to the description of following detailed exemplary embodiment, will be better understood the present invention itself, preferred embodiment and objects and advantages of the present invention, wherein:
Fig. 1 illustrates according to the method for stream processing of embodiments of the invention for distributed system;
Fig. 2 illustrates according to embodiments of the invention for flowing the distributed system of process;
Fig. 3 illustrates according to embodiments of the invention for flowing the distributed system of process;
Fig. 4 illustrates according to embodiments of the invention for flowing the distributed system of process; And
Fig. 5 diagrammatically illustrates the block diagram realizing computing equipment according to an embodiment of the invention.
Embodiment
Describe according to the method and system of the embodiment of the present invention based on communication message management work task below in conjunction with accompanying drawing, will be better understood objects and advantages of the present invention.
Core concept of the present invention: application data being calculated to the stream process of Existence dependency relationship, in order to avoid carrying out a large amount of transfer of data between computing node, the result of calculation only next data block being calculated the previous data block relied on is transmitted between computing node, and the multiple calculation procedures each data block completed needed for stream process complete on single computing node, multiple computing node realizes the method for stream processing walked abreast, just can obtain from the internal memory and processor cache of computing node this locality like this and realize the mass data needed for calculation procedure that stream processes each.
Fig. 1 illustrates according to the method for stream processing of embodiments of the invention for distributed system, this distributed system comprises distributor and multiple computing node, the method comprises: in step S101, in response to the data block receiving distributor distribution, described computing node calls corresponding computational logic and calculates described data block and produce result of calculation; In step S102, described computing node only calculates to next data block the computing node that the described result of calculation of relevant part sends next data block place to.
Fig. 2 illustrates the distributed system of carrying out flowing process according to embodiments of the invention, and this distributed system comprises distributor and multiple computing node P 1, P 2, P 3... P n, distributor physically can be in different geographical position from multiple computing node, also can be positioned at the data center in same geographical position.Distributor, as the interface of data flow and computing node, is responsible for the reception of data flow and the traffic monitoring of each computing node.The data flow received is divided into data block d one by one by distributor 1, d 2, d 3... ..d n, then distribute to available computing node.
In step S101, in response to the data block receiving distributor distribution, described computing node calls corresponding computational logic and calculates described data block and produce result of calculation.According to one embodiment of present invention, comprise in response to determining that described computing node can be used in taking a step forward of step S101, described computing node is to described distributor request dispatching data block.Particularly, local memory space behaviour in service and processor utilization rate can be inquired about by computing node, and pre-set the threshold value of storage size and processor utilization rate, judge that whether local memory space is sufficient by storage size available for this locality and threshold value being compared, and whether processor utilization rate and threshold value are compared decision processor busy, if have sufficient memory space according to judged result determination computing node and processor can be used, then request distributor distribute data.According to another embodiment of the invention, can determine whether computing node can be used by distributor, namely memory space behaviour in service and processor utilization rate can be reported by each computing node to distributor, also memory space behaviour in service and the processor utilization rate of all computing nodes regularly can be inquired about by distributor, the threshold value of storage size and processor utilization rate is pre-set at distributor place, then by the free memory size of computing node and processor utilization rate and threshold value being compared, distributor judges whether computing node can be used, and if determine that computing node can be used, then to the computing node order-assigned data block with sufficient storage space.
In response to the data block receiving distributor distribution, described computing node calls corresponding computational logic and calculates described data block and produce result of calculation.In stream processing procedure, be divided into multiple computational logic C according to the execution sequence of calculation procedure 1, C 2, C 3... .C nfor the same data block that computing node receives, need to carry out whole logical calculated according to the execution sequence of stream process calculation procedure, and front and back computational logic has dependence, a rear computational logic carries out based on the result of previous computational logic.And for adjacent data block, next data block carries out the result of calculation that logical calculated needs to depend on previous data block.
According to embodiments of the invention, comprise described computing node further and be previously stored with multiple computational logic, computing node calls corresponding computational logic according to the execution sequence of stream process calculation procedure and data block is calculated and produces result of calculation.First for the firm data block received from distributor, owing to also not carrying out any calculation procedure, therefore call initial calculation logic calculate data block and produce result of calculation, then call remaining computational logic according to the execution sequence of stream process calculation procedure and data are calculated.
According to another embodiment of the present invention, wherein said computational logic is distributed by described distributor.Described computing node comprises further: the state recording described data block; In response to detecting that the state of described data block changes, described computing node notifies the current state of data block described in described distributor; Distributor distributes corresponding computational logic according to the current state of described data block to described computing node.Particularly, by the state of each computing node recording data blocks, the state of data block comprises the current residing calculation procedure of data block, the state of recording data blocks such as can be carried out by arranging marker bit, when completing certain computational logic to data block, the state of data block changes, in response to detecting that the state of data block changes, the current state of computing node toll offering distributor data block, distributor distributes corresponding computational logic according to the current state of data block to computing node, computing node receives the computational logic of distribution, data block computation logic is calculated, by distributor Distribution Calculation logic, thus the whole computational logics realized on computing node a data block.
Each data block has needed to flow the multiple calculation procedures needed for process, the input of usual next calculation procedure not only needs the result of calculation of previous calculation procedure, also need the input of previous calculation procedure, the encryption of such as video flowing, therefore the multiple computational logics needing mass data to transmit are realized on a computing node, the result of previous calculation procedure and required data just can be obtained like this from the internal memory and processor cache of local computing node, the computational logic of next calculation procedure can directly process these data, thus avoid carry out transfer of data continually between computing node.
In step S102, described computing node only calculates to next data block the computing node that the described result of calculation of relevant part sends next data block place to.In a streaming process, for the application of the calculating Existence dependency relationship of data block, next data block calculates relevant to the some numerical results of previous data block, and therefore current computing node needs to calculate to next data block the computing node that the described result of calculation of relevant part sends next data block place to.Between computing node, realize the transfer of data of lightweight thus, avoid the transfer of data of big data quantity.
The distributed system of the stream process shown in composition graphs 3 describes one embodiment of the present of invention, multiple computing node P 1, P 2, P 3... P nbe previously stored with multiple computational logic C 1, C 2, C 3... ..C n, distributor receives data flow, and data flow is divided into multiple data block d 1, d 2, d 3... .., is determining computing node P 1, P 2and P 3available, to computing node P 1distribute data block d 1, to computing node P 2distribute data block d 2, and to computing node P 3distribute data block d 3.For convenience of explanation, multiple computational logic C is supposed 1, C 2, C 3... ..C nprocessing time roughly the same, be divided into multiple time period at this and be described.
First, at first time period T 1, computing node P 1first computational logic C is called according to the order of calculation procedure 1to data block d 1calculate, produce result of calculation R 1, will with data block d 2the relevant some numerical results R of calculating 1' send computing node P to 2.
Then, at second time period T 2, computing node P 2according to the sequence call computational logic C of calculation procedure 1to data block d 2and R 1' calculate, produce result of calculation S 1, and will with data block d 3the relevant S of calculating 1some numerical results S 1' send computing node P to 3.Meanwhile, computing node P 1according to the sequence call computational logic C of calculation procedure 2to computing node P 1on data block d 1' calculate, wherein d 1' comprise d 1and R 1, produce result of calculation R 2, will with computing node P 2on data block d 2' the relevant some numerical results R of calculating 2' send computing node P to 2.
At the 3rd time period T 3, computing node P 3receive computing node C 2the some numerical results R sent 2', according to the sequence call computational logic C of calculation procedure 1to data block d 3and S 1' calculate, produce result of calculation V 1, will with data block d 4the relevant some numerical results V of calculating 1' send data block d to 4the computing node at place.Meanwhile, computing node P 1according to the sequence call computational logic C of calculation procedure 3to data block d 1" calculate, produce result of calculation R 3, will with data block d 2the relevant some numerical results R of calculating 3' send computing node P to 2.Computing node P 2according to the sequence call computational logic C of calculation procedure 2to data block d 2' and R 3' calculate, produce result of calculation S 2, and will with data block d 3the relevant some numerical results S of calculating 2' send computing node P to 3.Each computing node completes according to calculation procedure and carries out flowing the whole computational logics needed for process to the data block of distributing, delete this computing node carries out flowing process generation total data to the data block of distributing afterwards, empty memory space, wait for that distributor distributes next data block.
Stream process distributed system shown in composition graphs 4 describes one embodiment of the present of invention, and distributor receives data flow, and data flow is divided into multiple data block d 1, d 2, d 3... .., is determining computing node P 1, P 2and P 3available, to computing node P 1distribute data block d 1, to computing node P 2distribute data block d 2, and to computing node P 3distribute data block d 3, for convenience of explanation, suppose multiple computational logic C 1, C 2, C 3... ..C nprocessing time roughly the same, be divided into multiple time period at this and be described.
First, at first time period T 1, computing node P 1, P 2and P 3state flag bit is set respectively and carrys out recording data blocks block d 1, d 2, d 3state be initial condition, because distributor knows for sure now data block d 1, d 2, d 3be in initial condition, therefore to computing node P 1, P 2and P 3distribution Calculation logic C respectively 1, computing node P 1call computational logic C 1to data block d 1calculate, produce result of calculation R 1, will with data block d 2the relevant some numerical results R of calculating 1' send computing node P to 2, now in response to computational logic C having been detected 1, computing node P 1state flag bit is revised as computational logic C 1, and toll offering distributor device data block d 1current state, distributor is according to data block d 1current state to computing node P 1distribution Calculation logic C 2.
At second time period T 2, computing node P 2call computational logic C 1to data block d 2and R 1' calculate, produce result of calculation S 1, and will with data block d 3the relevant S of calculating 1some numerical results S 1' send computing node P to 3, now in response to data block d being detected 2complete computational logic C 1, computing node P 2state flag bit is revised as data block d 2complete computational logic C 1, and toll offering distributor data block d 2current state, distributor is according to data block d 2current state to computing node P 2distribution Calculation logic C 2.Meanwhile, computing node P 1call computational logic C 2to computing node P 1on data block d 1' calculate, wherein d 1' comprise d 1and R 1, produce result of calculation R 2, will with computing node P 2on data block d 2' the relevant some numerical results R of calculating 2' send computing node P to 2.Now in response to computational logic C having been detected 2, computing node P 1state flag bit is revised as computational logic C 2, and toll offering distributor device data block d 1' current state, distributor is according to data block d 1' current state to computing node P 1distribution Calculation logic C 3.
At the 3rd time period T 3, computing node P 3receive computing node P 2the some numerical results R sent 2', call computational logic C 1to data block d 3and S 1' calculate, produce result of calculation V 1, will with data block d 4the relevant some numerical results V of calculating 1' send data block d to 4the computing node at place.Now in response to data block d being detected 3complete computational logic C 1, computing node P 3state flag bit is revised as data block d 3complete computational logic C 3, and toll offering distributor data block d 3current state, distributor is according to data block d 3current state to computing node P 3distribution Calculation logic C 2.Meanwhile, computing node P 1call computational logic C 3to computing node P 1on data block d 1" calculate, wherein d 1" comprise d 1' and R 2, produce result of calculation R 3, will with computing node P 2on data block d 2' the relevant some numerical results R of calculating 3' send computing node P to 2.Now in response to data block d being detected 1" complete computational logic C 3, computing node P 1state flag bit is revised as data block d 1" complete computational logic C 2, and toll offering distributor device data block d 1" current state, distributor is according to data block d 1" current state to computing node P 1distribution Calculation logic C 4.Computing node P 2call computational logic C 2to data block d 2' and R 1' calculate, produce result of calculation S 2, and will with data block d 3the relevant some numerical results S of calculating 2' send computing node P to 3, now in response to data block d being detected 2' complete computational logic C 2, computing node P 2state flag bit is revised as data block d 2' complete computational logic C 2, and toll offering distributor data block d 2' current state, distributor is according to data block d 2' current state to computing node P 2distribution Calculation logic C 3.Each computing node completes and carries out flowing the whole computational logics needed for process to the data block of distributing, delete this computing node carries out flowing process generation total data to the data block of distributing afterwards, empty memory space, wait for that distributor distributes next data block.
Based on same inventive concept, the present invention proposes a kind of distributed system of carrying out flowing process, and Fig. 2 illustrates that this distributed system comprises: distributor according to an embodiment of the invention for flowing the distributed system of process; Multiple computing node, is configured to the data block in response to receiving distributor distribution, calls corresponding computational logic and calculates described data block and produce result of calculation; And only calculate to next data block the computing node that the described result of calculation of relevant part sends next data block place to.
According to embodiments of the invention, wherein said computing node is configured to prestore multiple computational logic further, and wherein said computing node is configured to further: described computing node to calculate described data block according to the computational logic corresponding to sequence call of stream process calculation procedure and produces result of calculation.
According to embodiments of the invention, wherein said computational logic is distributed by described distributor.Wherein said computing node is configured to further: the state of data block described in described computing node record; In response to detecting that the state of described data block changes, described computing node notifies the current state of data block described in described distributor; Distributor distributes corresponding computational logic according to the current state of described data block to described computing node.
According to embodiments of the invention, wherein said computing node is configured in response to determining that described computing node can be used further, to described distributor request dispatching data block.
Fig. 5 then diagrammatically illustrates the block diagram that can realize computing equipment according to an embodiment of the invention.Computer system shown in Fig. 5 comprises CPU (CPU) 501, RAM (random access memory) 502, ROM (read-only memory) 503, system bus 504, hard disk controller 505, keyboard controller 506, serial interface controller 507, parallel interface controller 508, display controller 509, hard disk 510, keyboard 511, serial peripheral equipment 512, concurrent peripheral equipment 513 and display 514.In these parts, what be connected with system bus 504 has CPU 501, RAM 502, ROM 503, hard disk controller 505, keyboard controller 506, serial interface controller 507, parallel interface controller 508 and display controller 509.Hard disk 510 is connected with hard disk controller 505, keyboard 511 is connected with keyboard controller 506, serial peripheral equipment 512 is connected with serial interface controller 507, and concurrent peripheral equipment 513 is connected with parallel interface controller 508, and display 514 is connected with display controller 509.
In Fig. 5, the function of each parts is being well-known in the art, and the structure shown in Fig. 5 is also conventional.This structure not only for personal computer, and for handheld device, as Palm PC, PDA (personal digital assistant), mobile phone etc.In different applications, time such as realizing including according to the user terminal of client modules of the present invention or including according to network application server of the present invention server host, can add some parts to the structure shown in Fig. 5, or some parts in Fig. 5 can be omitted.Whole system shown in Fig. 5 is by being stored in hard disk 510 usually used as software or the computer-readable instruction be stored in EPROM or other nonvolatile memory controls.Software also can be downloaded from network (not shown).Or be stored in hard disk 510, or can be loaded into RAM 502 from the software of web download, and performed by CPU 501, to complete the function determined by software.
Although the computer system described in Fig. 5 can be supported according to technical scheme provided by the invention, this computer system is an example of computer system.It will be apparent to those skilled in the art that other Computer System Design many also can realize embodiments of the invention.
Although describe exemplary embodiment of the present invention here with reference to the accompanying drawings, but should be appreciated that and the invention is not restricted to these accurate embodiments, and when not deviating from scope of the present invention and aim, those of ordinary skill in the art can carry out the amendment of various change to embodiment.All such changes and modifications are intended to be included in scope of the present invention defined in the appended claims.
Should be appreciated that at least some aspect of the present invention can alternately realize with program product.The program defining function for the present invention can be sent to data-storage system or computer system by various signal bearing medium, described signal bearing medium includes but not limited to, storage medium can not be write (such as, CD-ROM) storage medium (such as, floppy disk, hard disk drive, read/write CD ROM, light medium), can be write and such as comprise the communication media of computer and telephone network and so on of Ethernet.Therefore it should be understood that in this type of signal bearing medium, when the computer-readable instruction of methodological function carried or encode in management the present invention, represent alternate embodiments of the present invention.The present invention can hardware, software, firmware or its mode combined realize.The present invention can realize in a concentrated manner in a single computer system, or realizes in a distributed fashion, and in this distribution mode, different component distribution is in the computer system of some interconnection.Any computer system or other device that are suitable for performing method described herein are all suitable.Preferably, the present invention realizes in the mode of the combination of computer software and common computer hardware, in this implementation, when being loaded and executed, control this computer system and make it perform method of the present invention, or forming system of the present invention.
Above for illustrational object, give the explanation of the preferred embodiments of the present invention.The above-mentioned explanation of preferred embodiment is not limit, does not intend the present invention to be confined to disclosed clear and definite form yet, and obviously in view of above-mentioned instruction, many modifications and variations are possible.Apparent this modifications and variations are included in the scope of the present invention that limited by the claim of adding to one skilled in the art.

Claims (10)

1., for a method for stream processing for distributed system, this distributed system comprises distributor and multiple computing node, and the method comprises:
In response to the data block receiving distributor distribution, described computing node calls corresponding computational logic and calculates described data block and produce result of calculation; And
Described computing node only calculates to next data block the computing node that the described result of calculation of relevant part sends next data block place to,
Comprise further:
The state of data block described in described computing node record;
In response to detecting that the state of described data block changes, described computing node notifies the current state of data block described in described distributor;
Distributor distributes corresponding computational logic according to the current state of described data block to described computing node.
2. method for stream processing according to claim 1, is included in described computing node further and is previously stored with multiple computational logic.
3. method for stream processing according to claim 2, wherein said computing node calls corresponding computational logic and calculates described data block and produce result of calculation and comprise:
Described computing node to calculate described data block according to the computational logic corresponding to sequence call of stream process calculation procedure and produces result of calculation.
4. method for stream processing according to claim 1, wherein said computational logic is distributed by described distributor.
5. the method for stream processing according to any one of claim 1-4, comprises further: in response to determining that described computing node can be used, and described computing node is to described distributor request dispatching data block.
6. carry out the distributed system flowing process, this distributed system comprises:
Distributor;
Multiple computing node, is configured to the data block in response to receiving distributor distribution, calls corresponding computational logic and calculates described data block and produce result of calculation; And only calculate to next data block the computing node that the described result of calculation of relevant part sends next data block place to,
Wherein said computing node is configured to further:
The state of data block described in described computing node record;
In response to detecting that the state of described data block changes, described computing node notifies the current state of data block described in described distributor;
Distributor distributes corresponding computational logic according to the current state of described data block to described computing node.
7. distributed system according to claim 6, wherein said computing node is configured to be previously stored with multiple computational logic further.
8. distributed system according to claim 7, wherein said computing node is configured to further:
Described computing node to calculate described data block according to the computational logic corresponding to sequence call of stream process calculation procedure and produces result of calculation.
9. distributed system according to claim 6, wherein said computational logic is distributed by described distributor.
10. the distributed system according to any one of claim 6-9, wherein said computing node is configured in response to determining that described computing node can be used further, to described distributor request dispatching data block.
CN201110214487.XA 2011-07-29 2011-07-29 Stream processing method and distributed system for realizing stream processing Expired - Fee Related CN102904919B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201110214487.XA CN102904919B (en) 2011-07-29 2011-07-29 Stream processing method and distributed system for realizing stream processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201110214487.XA CN102904919B (en) 2011-07-29 2011-07-29 Stream processing method and distributed system for realizing stream processing

Publications (2)

Publication Number Publication Date
CN102904919A CN102904919A (en) 2013-01-30
CN102904919B true CN102904919B (en) 2015-05-20

Family

ID=47576955

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201110214487.XA Expired - Fee Related CN102904919B (en) 2011-07-29 2011-07-29 Stream processing method and distributed system for realizing stream processing

Country Status (1)

Country Link
CN (1) CN102904919B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103595795B (en) * 2013-11-15 2016-10-05 北京国双科技有限公司 The treating method and apparatus of distributed data
CN105335376B (en) * 2014-06-23 2018-12-07 华为技术有限公司 A kind of method for stream processing, apparatus and system
CN105335218A (en) * 2014-07-03 2016-02-17 北京金山安全软件有限公司 Streaming computing method and streaming computing system based on local
CN107357639B (en) 2016-05-09 2019-09-17 腾讯科技(深圳)有限公司 A kind of distributed processing system(DPS), data processing method and apparatus
CN106874133B (en) 2017-01-17 2020-06-23 北京百度网讯科技有限公司 Failure handling for compute nodes in a streaming computing system
CN108595685B (en) * 2018-05-04 2021-06-15 北京顶象技术有限公司 Data processing method and device
CN108664331B (en) * 2018-05-22 2020-08-11 腾讯大地通途(北京)科技有限公司 Distributed data processing method and device, electronic equipment and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101072133A (en) * 2007-05-23 2007-11-14 华中科技大学 High-performance computing system based on peer-to-peer network

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101072133A (en) * 2007-05-23 2007-11-14 华中科技大学 High-performance computing system based on peer-to-peer network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
《An Ant Colony Optimization Approach to a Grid Workflow Scheduling Problem With Various Qos Requirements》;Wei-Neng Chen 等;《IEEE Transactions on Systems,Man,and Cybernetics-part C:Applications and Reviews》;20090131;第39卷(第1期);第29-43页 *
《基于进程通信的电力系统分布式计算模型》;文俊浩 等;《计算机工程与应用》;20090131;第45卷(第1期);第213-216页 *

Also Published As

Publication number Publication date
CN102904919A (en) 2013-01-30

Similar Documents

Publication Publication Date Title
CN102904919B (en) Stream processing method and distributed system for realizing stream processing
CN108509276B (en) Video task dynamic migration method in edge computing environment
KR101473317B1 (en) Cloud computing system and traffic distributing and control method in cloud computing system
KR102172169B1 (en) Application excution system based on distributed cloud, apparatus and control method thereof using the system
JP2010537318A (en) Media streaming with online cache and peer-to-peer transfer
US9712612B2 (en) Method for improving mobile network performance via ad-hoc peer-to-peer request partitioning
CN104038540A (en) Method and system for automatically selecting application proxy server
CN104396215A (en) Method and apparatus for bandwidth allocation and estimation
US20140188801A1 (en) Method and system for intelligent load balancing
US9374418B2 (en) Systems, methods and algorithms for logical movement of data objects
JP7006607B2 (en) Distributed processing system, distributed processing method, and recording medium
US11750711B1 (en) Systems and methods for adaptively rate limiting client service requests at a blockchain service provider platform
CN110933609A (en) Service migration method and device based on dynamic environment perception
CN113472844A (en) Edge computing server deployment method, device and equipment for Internet of vehicles
CN109618174A (en) A kind of live data transmission method, device, system and storage medium
CN105094981A (en) Method and device for processing data
KR20180118393A (en) Distributed cloud computing system, apparatus and control method thereof using the system
CN106797399A (en) Determine collocation method, equipment, system and the computer-readable medium of the new configuration of computing resource
CA2857727C (en) Computer-implemented method, computer system, computer program product to manage traffic in a network
CN113778675A (en) Calculation task distribution system and method based on block chain network
CN112152879B (en) Network quality determination method, device, electronic equipment and readable storage medium
CN105242955A (en) Virtual machine disk input and output scheduling method and system
CN102200928A (en) Computation resource control apparatus, computation resource control method, and non-transitory computer-readable recording medium
JP5725043B2 (en) Communication device and communication method
CN115033904A (en) Data processing method, apparatus, system, medium, and product

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20150520

Termination date: 20160729

CF01 Termination of patent right due to non-payment of annual fee