CN102214184A - Intermediate file processing device and intermediate file processing method of distributed computing system - Google Patents
Intermediate file processing device and intermediate file processing method of distributed computing system Download PDFInfo
- Publication number
- CN102214184A CN102214184A CN 201010146041 CN201010146041A CN102214184A CN 102214184 A CN102214184 A CN 102214184A CN 201010146041 CN201010146041 CN 201010146041 CN 201010146041 A CN201010146041 A CN 201010146041A CN 102214184 A CN102214184 A CN 102214184A
- Authority
- CN
- China
- Prior art keywords
- intermediate file
- file
- map
- reduce
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides an intermediate file processing device and an intermediate file processing method of a distributed computing system. Based on a Map-Reduce frame, the device comprises a Map unit and a Reduce unit, wherein the Map unit comprises an intermediate file generation module and a transmission module; the intermediate file generation module is used for generating a plurality of intermediate files according to a predetermined file size after processing a Map task; the transmission module sequentially transmits the intermediate files to the Reduce unit according to the generation sequence of the plurality of intermediate files; the Reduce unit comprises a communication module and a computation module; the communication module receives the intermediate files; and the computation module computes the intermediate files and outputs a final result. By adoption of the device and the method provided by the invention, the operating efficiency of the distributed computing system can be improved.
Description
[technical field]
The present invention relates to the distributed computing technology field, relate in particular to a kind of intermediate file treating apparatus and method of distributed computing system.
[background technology]
When distributed computing system (Map-Reduce) is handled big data quantity, huge data slicer is divided and rule, carry out the result after every data of parallel computation and gather.Traditional distributed computing system comprises Master unit, some Map unit and some Reduce unit.Wherein, the Master unit is the major control program of distributed computing system, is responsible for the task scheduling of Map unit and Reduce unit, the running status of controlling their operation and monitoring them; The Map unit is the unit of handling a part of data, and whole data are handled by a plurality of Map, and each Map can produce interim intermediate result (being intermediate file); Reduce is responsible for the unit intermediate result after all Map cell processing is merged, and obtains net result.
Usually, each computing unit in the Map-Reduce system (some Map unit and some Reduce unit) is deployed in some computing machine, because computing machine self local disk limited storage space, and the intermediate computations file is very big, surpass computing machine local disk space size, has only the 500M remaining space as local disk, and the intermediate data file that calculates has 100G even bigger, therefore the Reduce unit correctly need be preserved and be transferred to the intermediate data file of this magnanimity and further calculate.
Traditional mode be will operation Map unit computing machine big file cutting to be processed, be cut into a plurality of small documents, make following that the intermediate file local disk of generation can put, the Map unit number of Xu Yaoing is just many like this.Yet because the server cost is very high, our hardware resource all is limited usually, and when machine quantity fixedly the time, because big file has been divided into a plurality of small documents, each Map handles the file number that needs move and can increase.And for each Map unit, its processing procedure is serial, promptly must finish the processing that could start next file after the processing of current file, and owing to be subjected to the restriction of local disk space, whole M ap-Reduce task then needs the processing procedure of a very long time just can obtain final result, therefore, traditional this processing mode operational efficiency is not high.And this mode developer need consider the cutting of file etc., and is not knowing that being difficult to realization under the space operating position of disk carries out cutting to file according to suitable size, so underaction; And will handle big file the time, need a large amount of Map unit, and the hardware resource that needs is many, the cost height, and the complexity of realization and difficulty are all higher.
[summary of the invention]
Based on this, be necessary to provide a kind of intermediate file treating apparatus that can improve the distributed computing system of operational efficiency.
A kind of intermediate file treating apparatus of distributed computing system, described device is based on the Map-Reduce framework, comprise Map unit and Reduce unit, described Map unit comprises: the intermediate file generation module is used for generating a plurality of intermediate files according to default file size after handling the Map task; Transport module transfers to the Reduce unit with described intermediate file successively according to the genesis sequence of a plurality of intermediate files; Described Reduce unit comprises the communication module that receives described intermediate file, and the computing module that described intermediate file is calculated, exports net result.
Described Map unit also can comprise: the description document generation module, and the Map description document of the information of the described intermediate file of generation record, the information of described intermediate file comprises: file size, file name and file cryptographic hash; After described Reduce reads described intermediate file in the unit, obtain the information of described intermediate file, the information of described intermediate file is deposited in the Reduce description document.
This Map unit also can comprise: the Map detection module, detect the intermediate file number that generates according to described Map description document and whether surpass first threshold; The Map control module, when the intermediate file number that detects generation when described Map detection module surpassed first threshold, control suspended the generation of intermediate file.
And the Map control module also can be used for after the Map unit has transmitted an intermediate file, then deletes the intact intermediate file of this transmission.
Wherein, described Reduce unit also can comprise: the Reduce detection module is used for judging according to described Reduce description document whether the interim intermediate file number of preserving in Reduce unit surpasses second threshold value; The Reduce control module, when the intermediate file number that detects interim preservation when described Reduce detection module surpasses second threshold value, notify described Map unit to suspend the transmission of intermediate file, when the intermediate file number that detects interim preservation does not surpass second threshold value, notify described Map unit to continue the transmission intermediate file.
Wherein, described Reduce control module also can be used for the interim intermediate file of preserving in deletion Reduce unit after described computing module calculating finishes.
In addition, also be necessary to provide a kind of intermediate file disposal route that can improve the distributed computing system of operational efficiency.
A kind of intermediate file disposal route of distributed computing system, described method may further comprise the steps based on the Map-Reduce framework: handle the Map task, generate a plurality of intermediate files according to default file size; Genesis sequence according to a plurality of intermediate files transfers to intermediate file the Reduce unit successively; Described Reduce unit receives intermediate file, middle file is calculated the output net result.
The step of a plurality of intermediate files of described generation also can comprise the information that writes down described intermediate file by the Map description document, comprising: file size, file name and file cryptographic hash; The step that described Reduce unit receives intermediate file also comprises: obtain the information of intermediate file and deposit in the Reduce description document.
This method also can comprise: detect the intermediate file number that generates according to the Map description document and whether surpass first threshold, if then control suspends the generation of intermediate file.
Wherein, the genesis sequence according to a plurality of intermediate files also can comprise the step that intermediate file transfers to the Reduce unit: when having transmitted an intermediate file to the Reduce unit, then delete the intact intermediate file of described transmission.
This method also can comprise: judge according to described Reduce description document whether the interim intermediate file number of preserving in Reduce unit surpasses second threshold value, if, then suspend the transmission of intermediate file, otherwise, the transmission intermediate file continued.
And also can comprise the step that middle file calculates: the interim described intermediate file of preserving in deletion Reduce unit after the calculating of middle file finishes.
The intermediate file treating apparatus and the method for above-mentioned distributed computing system, by after handling the Map task, generating a plurality of intermediate files according to default file size, and successively it is transferred to the Reduce unit according to the genesis sequence of these intermediate files, be that the Map unit has the intermediate file generation then to be transferred to the Reduce unit, can improve the operational efficiency of distributed computing system like this.
In addition, the number threshold value of the size by intermediate file is set and the intermediate file of transmission, the quantity of intermediate file that can control transmission goes up in limited time when these intermediate file numbers reach certain, may command is suspended data transmission, and it is congested to have avoided too much data transmission to cause.For in the middle of solving in the distributed computing system during super large data file, not needing increases too much Map unit, thereby has saved hardware resource, and the complexity and the difficulty that realize all decrease.
[description of drawings]
Fig. 1 is the structural representation of the intermediate file treating apparatus of distributed computing system among the embodiment;
Fig. 2 is the structural representation of Map unit among the embodiment;
Fig. 3 is the structural representation of Map unit among another embodiment;
Fig. 4 is the structural representation of Reduce unit among the embodiment;
Fig. 5 is the structural representation of Reduce unit among another embodiment;
Fig. 6 is the process flow diagram of the intermediate file disposal route of distributed computing system among the embodiment.
[embodiment]
As shown in Figure 1, a kind of intermediate file treating apparatus of distributed computing system, comprise a plurality of Map unit 10 and a plurality of Reduce unit 20, wherein: the Map task is handled in each Map unit 10, receive input to (key/value) by the Map function, produce intermediate file and pass to Reduce unit 20, Reduce unit 20 receives a middle key and relevant value by the Reduce function, and merge these values, obtain final calculation result.
As shown in Figure 2, in one embodiment, Map unit 10 comprises intermediate file generation module 101 and transport module 102, and wherein: intermediate file generation module 101 is used for generating a plurality of intermediate files according to default file size after handling the Map task.Should default file size can be provided with according to the size of local disk, it is 100M that an intermediate file for example is set.Transport module 102 is used for transmitting it to Reduce unit 20 successively according to the genesis sequence of a plurality of intermediate files.
In another embodiment, as shown in Figure 3, Map unit 10 also comprises description document generation module 103, Map detection module 104 and Map control module 105 except comprising above-mentioned intermediate file generation module 101 and transport module 102, wherein:
Description document generation module 103 is used to generate the Map description document of the information that writes down intermediate file.During Map output intermediate file, can detect whether the Map description document is arranged in the local disk,, then generate this description document if do not have.This description document has write down the information of intermediate file, and the information of this intermediate file comprises the information such as file size, file name and file cryptographic hash of the intermediate file that is generated.Among this embodiment, the Map description document is intermediate_head.dat, when middle file generating module 101 generated first intermediate file (as intermediate_data-1.data) according to default file size, then the Map description document write down the information such as cryptographic hash of length, title and the file data of this document.When the intermediate file of output during greater than default file size (as 100M), then follow-up data continues to save as new intermediate file, as intermediate_data-2.data, and upgrade the Map description document, the rest may be inferred according to this principle for the intermediate file of follow-up output.Generate an intermediate file, this intermediate file that transport module 102 will generate at once is transferred to Reduce unit 20, i.e. the intermediate file that generates earlier transmission earlier.
In one embodiment, as shown in Figure 4, Reduce unit 20 comprises communication module 201 and computing module 202, and wherein: communication module 201 receives Map unit 10 and transmits the intermediate file of coming; 202 pairs of middle files of computing module calculate, export net result.Among this embodiment, Reduce unit 20 receives intermediate file, then read intermediate file, obtain the information of intermediate file, the information of the intermediate file that is obtained can be the cryptographic hash of intermediate file etc., the intermediate file information of obtaining (is comprised file size, information such as file name and file cryptographic hash) be stored in the Reduce description document, simultaneously, the intermediate file that calculates can be stored in Reduce unit 20 temporarily, carry out the computation process in the Reduce unit 20 then, after calculating finishes, interim intermediate file of preserving of deletion and the intermediate computations file that produces in computation process etc. allow Reduce unit 20 be in when next intermediate file is waited in Reduce unit 20 and wait for or halted state.
In another embodiment, as shown in Figure 5, Reduce unit 20 also comprises Reduce detection module 203 and Reduce control module 204 except comprising above-mentioned communication module 201, computing module 202.Wherein: Reduce detection module 203 judges that according to the Reduce description document whether Reduce unit 20 interim intermediate file numbers of preserving surpass second threshold value (as 2), if surpass, then show 10 transmission of Map unit too much.Reduce control module 204 is used for when the intermediate file number that detects preservation surpasses second threshold value, then notify Map unit 10 to suspend the transmission of intermediate file, when the intermediate file of 20 interim storages is less than second threshold value up to the Reduce unit, just notify Map unit 10 to continue transmission.Reduce control module 204 also is used for after calculating finishes, the deletion Reduce unit 20 interim intermediate files of preserving.
Behind all intermediate file end of transmissions of Map unit 10 generations, can notify Reduce unit 20 data transmission to finish, and in the Map description document, indicate the intermediate file end of transmission.
As shown in Figure 6, a kind of intermediate file disposal route of distributed computing system may further comprise the steps:
Step S10 handles the Map task, generates a plurality of intermediate files according to default file size.In one embodiment, the size of intermediate file can be set according to the local disk size, the size that each intermediate file for example is set is 100M all, and the intermediate file that is generated writes down its information by the Map description document, comprises file size, file name and cryptographic hash etc.Among this embodiment, can detect earlier whether the Map description document is arranged in the local disk, if do not have, then generation is used to write down the intermediate file information Map description document of (comprising file size, file name and file cryptographic hash etc.).When the intermediate file of output during greater than default file size (as 100M), generate the intermediate file of first 100M after, follow-up data saves as new intermediate file, the rest may be inferred.
Step S20 transfers to intermediate file the Reduce unit successively according to the genesis sequence of a plurality of intermediate files.In one embodiment, Map unit 10 generates an intermediate file, then at once the intermediate file that generates is passed to Reduce unit 20, i.e. the intermediate file that generates earlier transmission earlier.When having transmitted an intermediate file, then delete the intact intermediate file of this transmission, for local disk is saved the space to Reduce unit 20.
Step S30, described Reduce unit receives intermediate file, middle file is calculated the output net result.In one embodiment, after Reduce unit 20 receives intermediate file, read intermediate file, and the cryptographic hash of calculating intermediate file, with the information stores such as cryptographic hash of intermediate file in the Reduce description document, and intermediate file can be stored in the Reduce unit 20 temporarily, after the computation process of intermediate file finishes, then interim intermediate file of preserving of deletion and the intermediate computations file that produces in computation process etc. allow Reduce unit 20 be in when waiting for next intermediate file and wait for or halted state.
In one embodiment, this method comprises also whether the intermediate file number that generates according to Map description document detection Map unit 10 surpasses the step of first threshold.Among this embodiment, can set first threshold (as 2) in advance, when middle file number surpasses this first threshold, the travelling speed that then shows Map unit 10 is too fast, need this moment control to suspend the generation of intermediate file, when the number for the treatment of intermediate file is no more than this first threshold, continue to calculate to generate intermediate file.
In another embodiment, this method also comprises according to the Reduce description document and judges whether the intermediate file number of preserving the Reduce unit surpasses the step of second threshold value.Among this embodiment, can set second threshold value (as 2) in advance, when the Reduce unit 20 interim intermediate files of preserving surpass this second threshold value, say and show 10 transmission of Map unit too much, at this moment, need to suspend the transmission of intermediate file, when the 20 interim intermediate files of preserving are no more than this second threshold value up to the Reduce unit, just continue the transmission intermediate file, for the local disk of Reduce unit 20 is saved the space.
Should be noted that, said method and system also can combine with distributed memory system, after Map unit 10 intermediate files in the distributed computing system generate, upload in the distributed memory system earlier and store, Reduce unit 20 is downloaded from distributed memory system again.
The intermediate file treating apparatus and the method for above-mentioned distributed computing system, by after handling the Map task, generating a plurality of intermediate files according to default file size, and successively it is transferred to Reduce unit 20 according to the genesis sequence of these intermediate files, be that Map unit 10 has the intermediate file generation then to be transferred to the Reduce unit, can improve the operational efficiency of distributed computing system like this.
In addition, the number threshold value of the size by intermediate file is set and the intermediate file of transmission, the quantity of intermediate file that can control transmission goes up in limited time when these intermediate file numbers reach certain, may command is suspended data transmission, and it is congested to have avoided too much data transmission to cause.For in the middle of solving in the distributed computing system during super large data file, not needing increases too much Map unit 10, thereby has saved hardware resource, and the complexity and the difficulty that realize all decrease.
The above embodiment has only expressed several embodiment of the present invention, and it describes comparatively concrete and detailed, but can not therefore be interpreted as the restriction to claim of the present invention.Should be pointed out that for the person of ordinary skill of the art without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be as the criterion with claims.
Claims (12)
1. the intermediate file treating apparatus of a distributed computing system, described device is based on the Map-Reduce framework, comprise Map unit and Reduce unit, it is characterized in that, described Map unit comprises: the intermediate file generation module is used for generating a plurality of intermediate files according to default file size after handling the Map task; Transport module transfers to the Reduce unit with described intermediate file successively according to the genesis sequence of a plurality of intermediate files; Described Reduce unit comprises the communication module that receives described intermediate file, and the computing module that described intermediate file is calculated, exports net result.
2. the intermediate file treating apparatus of distributed computing system according to claim 1 is characterized in that, described Map unit also comprises:
The description document generation module, the Map description document of the information of the described intermediate file of generation record, the information of described intermediate file comprises: file size, file name and file cryptographic hash;
After described Reduce reads described intermediate file in the unit, obtain the information of described intermediate file, the information of described intermediate file is deposited in the Reduce description document.
3. the intermediate file treating apparatus of distributed computing system according to claim 2 is characterized in that, described Map unit also comprises:
The Map detection module detects the intermediate file number that generates according to described Map description document and whether surpasses first threshold;
The Map control module, when the intermediate file number that detects generation when described Map detection module surpassed first threshold, control suspended the generation of intermediate file.
4. the intermediate file treating apparatus of distributed computing system according to claim 3 is characterized in that, described Map control module also is used for after the Map unit has transmitted an intermediate file, then deletes the intact intermediate file of this transmission.
5. the intermediate file treating apparatus of distributed computing system according to claim 2 is characterized in that, described Reduce unit also comprises:
The Reduce detection module is used for judging according to described Reduce description document whether the interim intermediate file number of preserving in Reduce unit surpasses second threshold value;
The Reduce control module, when the intermediate file number that detects interim preservation when described Reduce detection module surpasses second threshold value, notify described Map unit to suspend the transmission of intermediate file, when the intermediate file number that detects interim preservation does not surpass second threshold value, notify described Map unit to continue the transmission intermediate file.
6. the intermediate file treating apparatus of distributed computing system according to claim 5 is characterized in that, described Reduce control module also is used for the interim intermediate file of preserving in deletion Reduce unit after described computing module calculating finishes.
7. the intermediate file disposal route of a distributed computing system, described method may further comprise the steps based on the Map-Reduce framework:
Handle the Map task, generate a plurality of intermediate files according to default file size;
Genesis sequence according to a plurality of intermediate files transfers to intermediate file the Reduce unit successively;
Described Reduce unit receives intermediate file, middle file is calculated the output net result.
8. the intermediate file disposal route of distributed computing system according to claim 7, it is characterized in that, the step of a plurality of intermediate files of described generation also comprises the information that writes down described intermediate file by the Map description document, comprising: file size, file name and file cryptographic hash; The step that described Reduce unit receives intermediate file also comprises: obtain the information of intermediate file and deposit in the Reduce description document.
9. the intermediate file disposal route of distributed computing system according to claim 8 is characterized in that, described method also comprises:
Detect the intermediate file number that generates according to the Map description document and whether surpass first threshold, if, then
Control suspends the generation of intermediate file.
10. the intermediate file disposal route of distributed computing system according to claim 7, it is characterized in that, described genesis sequence according to a plurality of intermediate files also comprises the step that intermediate file transfers to the Reduce unit: when having transmitted an intermediate file to the Reduce unit, then delete the intact intermediate file of described transmission.
11. the intermediate file disposal route of distributed computing system according to claim 8 is characterized in that, described method also comprises:
Judge according to described Reduce description document whether the interim intermediate file number of preserving in Reduce unit surpasses second threshold value, if, then suspend the transmission of intermediate file, otherwise, the transmission intermediate file continued.
12. the intermediate file disposal route of distributed computing system according to claim 8, it is characterized in that the described step that middle file is calculated also comprises: the interim described intermediate file of preserving in deletion Reduce unit after the calculating of middle file finishes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010146041 CN102214184B (en) | 2010-04-07 | 2010-04-07 | Intermediate file processing device and intermediate file processing method of distributed computing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010146041 CN102214184B (en) | 2010-04-07 | 2010-04-07 | Intermediate file processing device and intermediate file processing method of distributed computing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102214184A true CN102214184A (en) | 2011-10-12 |
| CN102214184B CN102214184B (en) | 2013-08-14 |
Family
ID=44745499
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010146041 Active CN102214184B (en) | 2010-04-07 | 2010-04-07 | Intermediate file processing device and intermediate file processing method of distributed computing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102214184B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105430078A (en) * | 2015-11-17 | 2016-03-23 | 浪潮(北京)电子信息产业有限公司 | Distributed storage method of mass data |
| CN105955819A (en) * | 2016-04-18 | 2016-09-21 | 中国科学院计算技术研究所 | Data transmission method and system based on Hadoop |
| WO2017005116A1 (en) * | 2015-07-08 | 2017-01-12 | 阿里巴巴集团控股有限公司 | Intermediate file processing method, clients, servers and system |
| WO2017031961A1 (en) * | 2015-08-24 | 2017-03-02 | 华为技术有限公司 | Data processing method and apparatus |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040073665A1 (en) * | 2002-10-11 | 2004-04-15 | Fujitsu Limited | Log management apparatus and storage medium |
| CN101022612A (en) * | 2006-02-16 | 2007-08-22 | 杭州波导软件有限公司 | Mobile terminal user information data duplicate method |
| CN101582064A (en) * | 2008-05-15 | 2009-11-18 | 阿里巴巴集团控股有限公司 | Method and system for processing enormous data |
-
2010
- 2010-04-07 CN CN 201010146041 patent/CN102214184B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040073665A1 (en) * | 2002-10-11 | 2004-04-15 | Fujitsu Limited | Log management apparatus and storage medium |
| CN101022612A (en) * | 2006-02-16 | 2007-08-22 | 杭州波导软件有限公司 | Mobile terminal user information data duplicate method |
| CN101582064A (en) * | 2008-05-15 | 2009-11-18 | 阿里巴巴集团控股有限公司 | Method and system for processing enormous data |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017005116A1 (en) * | 2015-07-08 | 2017-01-12 | 阿里巴巴集团控股有限公司 | Intermediate file processing method, clients, servers and system |
| US11500812B2 (en) | 2015-07-08 | 2022-11-15 | Alibaba Group Holding Limited | Intermediate file processing method, client, server, and system |
| WO2017031961A1 (en) * | 2015-08-24 | 2017-03-02 | 华为技术有限公司 | Data processing method and apparatus |
| CN105430078A (en) * | 2015-11-17 | 2016-03-23 | 浪潮(北京)电子信息产业有限公司 | Distributed storage method of mass data |
| CN105430078B (en) * | 2015-11-17 | 2019-03-15 | 浪潮(北京)电子信息产业有限公司 | A kind of distributed storage method of mass data |
| CN105955819A (en) * | 2016-04-18 | 2016-09-21 | 中国科学院计算技术研究所 | Data transmission method and system based on Hadoop |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102214184B (en) | 2013-08-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104021069B (en) | The management method and system of the software performance test based on distributed virtual machine system | |
| CN104092719B (en) | Document transmission method, device and distributed cluster file system | |
| CN106469085B (en) | The online migration method, apparatus and system of virtual machine | |
| CN102214184B (en) | Intermediate file processing device and intermediate file processing method of distributed computing system | |
| CN105677615B (en) | A kind of distributed machines learning method based on weka interface | |
| CN106201443B (en) | A kind of method and system based on the mostly concurrent written document of Storm streaming computing frame | |
| CN106682004A (en) | Redis Key management method and system | |
| CN107943572B (en) | Data migration method, device, computer equipment and storage medium | |
| CN107451012A (en) | A kind of data back up method and stream calculation system | |
| JP2010211546A (en) | Virtual machine management system, virtual machine management method, virtual machine management device and virtual machine management program | |
| CN107295045A (en) | A kind of message treatment method and device | |
| CN102821164A (en) | Efficient parallel-distribution type data processing system | |
| CN110493302A (en) | A kind of document transmission method, equipment and computer readable storage medium | |
| CN104881467A (en) | Data correlation analysis and pre-reading method based on frequent item set | |
| CN103049421B (en) | Data transmission method and device between a kind of CPU and coprocessor | |
| CN108090186A (en) | A kind of electric power data De-weight method on big data platform | |
| CN104251913B (en) | Rotating machinery monitors system start and stop intelligent determination method | |
| CN113672410A (en) | Data processing method and electronic device | |
| CN108491483B (en) | Method and device for remotely monitoring running state of program | |
| CN107786670A (en) | A kind of system and method that data and cloud storage R. concomitans are locally stored | |
| CN102546734B (en) | Data information processing system and method | |
| CN111352810A (en) | Performance monitoring method and device based on multiple processes and computing equipment | |
| CN113626151B (en) | Container cloud log collection resource control method and system | |
| CN107967172A (en) | A kind of Spark tasks dynamic migration method and system towards isomery storage | |
| EP3396553B1 (en) | Method and device for processing data after restart of node |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: SHENZHEN SHIJI LIGHT SPEED INFORMATION TECHNOLOGY Free format text: FORMER OWNER: TENGXUN SCI-TECH (SHENZHEN) CO., LTD. Effective date: 20131015 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 518044 SHENZHEN, GUANGDONG PROVINCE TO: 518057 SHENZHEN, GUANGDONG PROVINCE |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20131015 Address after: A Tencent Building in Shenzhen Nanshan District City, Guangdong streets in Guangdong province science and technology 518057 16 Patentee after: Shenzhen Shiji Guangsu Information Technology Co., Ltd. Address before: Shenzhen Futian District City, Guangdong province 518044 Zhenxing Road, SEG Science Park 2 East Room 403 Patentee before: Tencent Technology (Shenzhen) Co., Ltd. |