CN102855297A - Method for controlling data transmission, and connector - Google Patents
Method for controlling data transmission, and connector Download PDFInfo
- Publication number
- CN102855297A CN102855297A CN2012102892131A CN201210289213A CN102855297A CN 102855297 A CN102855297 A CN 102855297A CN 2012102892131 A CN2012102892131 A CN 2012102892131A CN 201210289213 A CN201210289213 A CN 201210289213A CN 102855297 A CN102855297 A CN 102855297A
- Authority
- CN
- China
- Prior art keywords
- load information
- connector
- data transmission
- server
- threshold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a method for controlling data transmission, and a connector, and belongs to the field of data transmission. The method comprises the following steps that: when Hadoop transmits data to a relational database, the connector acquires load information of a server where the relational database is positioned; and the connector controls data transmission according to the load information. The connector comprises an acquisition module and a control module. By the method and the connector, influence caused by other services on the relational database during data transmission is reduced.
Description
Technical field
The present invention relates to field of data transmission, particularly a kind of method of control data transmission and connector.
Background technology
Along with enterprise will store with the data volume of analyzing and processing increasingly, Hadoop more and more comes into one's own, and Hadoop is a distributed system architecture, is the project of increasing income of Apache Software Foundation.Because Hadoop has the advantage that can not be substituted at scalability, robustness, calculated performance and cost, in fact become large data storage and the analysis platform of current main-stream.Storage is by HDFS(Hadoop Distributed File System, Hadoop distributed file system) realize, process and realized by MapReduce.
Relevant database and Hadoop can carry out data by connector to be shifted mutually, data in the relevant database can be led HDFS or the HBase(Hadoop Database that enters Hadoop by connector, the Hadoop database) in, also the data of HDFS or HBase can be led entering in the relevant database.
Sqoop is a kind of connector of increasing income that Cloudera company provides, and we can carry out by order line importing or the derivation of data to utilize Sqoop, the principle of below carrying out data importing and derivation for Sqoop:
(1) Sqoop imports data from relevant database to Hadoop
Sqoop is from relevant database to Hadoop importing table by a MapReduce operation, wherein, each MapReduce operation can comprise a plurality of Map tasks, and this MapReduce operation is extracted record from the table of relevant database, then record is write HDFS.
Fig. 1 has demonstrated Sqoop and how to have carried out mutual with source database and Hadoop.Before importing beginning, Sqoop at first checks the table that will import in the relevant database, it retrieves the SQL(Structured Query Language of row all in the table and row, Structured Query Language (SQL)) data type, these SQL types are mapped to the Java data type, in the MapReduce operation, will preserve with these corresponding java class types the value of field, the code generator of Sqoop creates the class of corresponding table with these information, these classes be used for to be preserved the record that extracts from table, then a plurality of Map tasks by a MapReduce operation with the data importing in the relevant database in the HDFS of Hadoop.
(2) Sqoop imports data from Hadoop to relevant database
Referring to Fig. 2, before importing beginning, Sqoop need to check the table in the relevant database equally, and then the definition according to object table in the relevant database generates a java class, the class of this generation can be separated new record from text, and can be to the value of insertion respective type in the object table.Then can start a MapReduce operation, by reading source data file among the HDFS of a plurality of Map tasks from Hadoop in the MapReduce operation, then use the class solution new record that generates, and carry out selected deriving method, with the data importing among the HDFS of Hadoop in relevant database.
In realizing process of the present invention, the inventor finds that there is following problem at least in prior art:
Existing connector can bring very heavy load to relevant database during the transmission of data between Hadoop and relevant database, and these relevant databases may carry other business simultaneously, therefore, the data transmission of connector goes out and can other business of relevant database be affected, particularly in the scene of carrying out the frequent transmission data.
Summary of the invention
The impact that other business of relevant database has been caused during data transmission between Hadoop and relevant database in order to have reduced connector, the embodiment of the invention provides a kind of method and connector of control data transmission.Described technical scheme is as follows:
On the one hand, provide a kind of method of control data transmission, described method comprises:
When Hadoop and relevant database carried out data transmission, connector obtained the load information of the server at described relevant database place;
Described connector is controlled described data transmission according to described load information.
Wherein, connector obtains the load information of the server at described relevant database place, comprising:
Described connector receives the load information that the monitor agent on the described server sends;
Perhaps, described connector is surveyed the load information that obtains described server.
Wherein, described connector is controlled described data transmission according to described load information, comprising:
Described connector judges according to described load information whether described server is overload state;
If so, then stop described data transmission; Otherwise, continue described data transmission.
Wherein, described connector judges that according to described load information whether described server is overload state, comprising:
Described connector compares described load information and the first threshold of presetting;
If described load information, determines then that described server has reached overload state more than or equal to described first threshold;
If described load information, determines then that described server does not reach overload state less than described first threshold.
Wherein, determine that described server does not reach after the overload state, also comprises:
Described connector compares described load information and the Second Threshold of presetting;
If described load information more than or equal to described the second threshold values, then reduces the speed of described data transmission;
If described load information less than described the second threshold values, then improves the speed of described data transmission.
Wherein, the above-described method of any one, described load information comprises following at least a: the central processing unit load information of described server, input and output load information and current EMS memory occupation information.
On the other hand, provide a kind of connector, described connector comprises: acquisition module and control module;
Described acquisition module is used for obtaining the load information of the server at described relevant database place when Hadoop and relevant database carry out data transmission;
Described control module is used for controlling described data transmission according to described load information.
Wherein, described acquisition module comprises:
Receiving element is used for receiving the load information that the monitor agent on the described server sends;
Perhaps, probe unit is used for surveying the load information that obtains described server.
Wherein, described control module comprises:
Judging unit is used for judging according to described load information whether described server is overload state;
Performance element is yes if be used for the result of described judging unit, then stops described data transmission; Otherwise, continue described data transmission.
Wherein, described judging unit is used for:
Described load information and the first threshold of presetting are compared;
If described load information, determines then that described server has reached overload state more than or equal to described first threshold;
If described load information, determines then that described server does not reach overload state less than described first threshold.
Wherein, described judging unit also is used for:
After described server does not reach overload state, described load information and the Second Threshold of presetting are compared;
If described load information more than or equal to described the second threshold values, then reduces the speed of described data transmission;
If described load information less than described the second threshold values, then improves the speed of described data transmission.
Wherein, the above-described connector of any one, described load information comprises following at least a: the central processing unit load information of described server, input and output load information and current EMS memory occupation information.
The beneficial effect that the technical scheme that the embodiment of the invention provides is brought is:
By when Hadoop and relevant database carry out data transmission, connector obtains the load information of the server at relevant database place, then according to the load information control data transmission, reduced the impact that connector causes other business of relevant database when data transmission.
Description of drawings
In order to be illustrated more clearly in the technical scheme in the embodiment of the invention, the accompanying drawing of required use was done to introduce simply during the below will describe embodiment, apparently, accompanying drawing in the following describes only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
Fig. 1 be prior art provide utilize Sqoop with the synoptic diagram of the data importing Hadoop in the relevant database;
Fig. 2 be prior art provide utilize Sqoop with the synoptic diagram of the data importing relevant database among the Hadoop;
Fig. 3 is the data flow figure with the data importing Hadoop in the relevant database provided by the invention;
Fig. 4 is the data flow figure with the data importing relevant database among the Hadoop provided by the invention;
Fig. 5 is the process flow diagram of the method for a kind of control data transmission of providing of the embodiment of the invention one;
Fig. 6 is the process flow diagram of the method for a kind of control data transmission of providing of the embodiment of the invention two;
Fig. 7 is the synoptic diagram of a kind of connector control data transmission of providing of the embodiment of the invention two;
Fig. 8 is the process flow diagram of the method for a kind of control data transmission of providing of the embodiment of the invention three;
Fig. 9 is the synoptic diagram of a kind of connector control data transmission of providing of the embodiment of the invention three;
Figure 10 is the structural drawing of a kind of connector of providing of the embodiment of the invention four;
Figure 11 is the structural drawing of a kind of control module of providing of the embodiment of the invention four.
Embodiment
For making the purpose, technical solutions and advantages of the present invention clearer, embodiment of the present invention is described further in detail below in conjunction with accompanying drawing.
Data transmission between Hadoop and the relevant database comprises the data importing in the relevant database in Hadoop, perhaps with the data importing among the Hadoop in relevant database.Wherein, referring to Fig. 3, data communication device in the relevant database is crossed the Map task and is imported among the HDFS among the Hadoop, each Map task is responsible for a database table (the perhaps part of a database table) in the relevant database is imported among the HDFS, and Hadoop can carry out a plurality of Map tasks simultaneously; Referring to Fig. 4, data among the Hadoop import in the relevant database by a plurality of Map tasks equally, each Map task is responsible for the transmission of a database table (the perhaps part of a database table), and Hadoop can carry out a plurality of Map tasks simultaneously.
Embodiment one
Referring to Fig. 5, present embodiment provides a kind of method of control data transmission, and described method comprises:
Step 101: when Hadoop and relevant database carried out data transmission, connector obtained the load information of the server at relevant database place;
Step 102: connector is according to the load information control data transmission.
The method of the control data transmission that present embodiment provides, by when Hadoop and relevant database carry out data transmission, connector obtains the load information of the server at relevant database place, then according to the load information control data transmission, reduced the impact that connector causes other business of relevant database when data transmission.
Embodiment two
The embodiment of the invention provides a kind of method of control data transmission, and referring to Fig. 6, described method comprises:
Step 201: connector receives user's input message;
Connector receives the order that the user carries out data transmission, comprising: with the data importing among the Hadoop of appointment in the relevant database of appointment, perhaps, with the data importing in the relevant database of appointment in the Hadoop of appointment.
Step 202: connect the information of obtaining the relevant database of appointment in the input message according to input message, and the generated data transformation task;
Connector can be accessed the data that are stored in the relevant database by JDBC, and checks character and the type of data.
For example, when Hadoop was arrived in the data importing in the relevant database, connector at first used JDBC to check the tables of data that will import, and retrieves the SQL type of row all in the tables of data and row; When the relevant database, connector at first needs to check character and the type of relevant database with the data importing among the Hadoop, generates corresponding class according to the definition of the table in the relevant database.
Step 203: connector sends to Hadoop with data transfer task, carries out data transmission between the Hadoop that begins in appointment and the relevant database of appointment;
Connector sends to Hadoop with data transfer task, Hadoop generates corresponding MapReduce operation according to data transfer task, wherein, a data transfer task obtains a MapReduce operation, each MapReduce operation is then finished in parallel mode by a plurality of map tasks, each Map task is responsible for transmitting a database table (the perhaps part of a database table), and a Hadoop server can be carried out a plurality of Map tasks simultaneously.
Step 204: connector receives the load information that the monitor agent on the server at relevant database place sends:
Load information can comprise following at least a: the central processing unit load information of server, input and output load information and current EMS memory occupation information, when if the load information of server comprises multiple load information, can be with a load information as current server of currency maximum in the load information of all kinds, for example, the load information of server comprises following three kinds: the central processing unit load information of server, input and output load information and current EMS memory occupation information, and the currency of three kinds of load informations is respectively: ten Percent, 20 percent and 30 percent, then the load information of current server is 30 percent.Server deploy monitor agent that can be on relational data periodically obtains the load information of server, then by monitor agent load information is sent connector.
Referring to Fig. 7, connector at first obtains the relational data library information, check character and the type of relational data database data, then submit job is to Hadoop, Hadoop carries out data transmission by carrying out a plurality of Map tasks, each Map task is responsible for the transmission of a database table (the perhaps part of a database table), connector receives the load information that the monitor agent on the server at relevant database place sends, and then controls the control that each Map task realizes data transmission according to load information.
Step 205: connector judges according to load information whether described server is overload state, if then execution in step 206, otherwise, execution in step 207;
Wherein, connector adopts following concrete mode to judge whether server is overload state according to load information:
Connector compares load information and the first threshold of presetting;
If load information, determines then that server has reached overload state more than or equal to first threshold;
If load information, determines then that server does not reach overload state less than first threshold.
The user can specify arbitrarily numerical value to the first threshold values.For example, the user can be made as 90% with the first threshold values, if the load information of server, determines that server has reached overload state more than or equal to 90%; If load information, determines then that server does not reach overload state less than 90%.
Step 206: stop data transmission, flow process finishes;
Step 207: continue data transmission, flow process finishes;
Wherein, step 207 can also comprise:
Connector compares described load information and the Second Threshold of presetting, if load information more than or equal to described the second threshold values, reduces the speed of data transmission, if load information less than described the second threshold values, improves the speed of data transmission.
Then the different load informations constantly of record server carry out statistical study to all different load informations constantly that record obtains, and obtain the second threshold values according to the result of statistical study.In addition, the second threshold values also can be specified by the user, and the user can arrange concrete numerical value to the second threshold values according to actual conditions.
For example, the speed of current data transmission is V
1, load information is compared with the second threshold values, if load information greater than the second threshold values, can be with the speed of current data transmission from V
1Be down to V
2, and V
2=90%V
1If, when the speed of data transmission is V
2The time server load information still more than or equal to the second threshold values, then continue to reduce the speed of data transmission; If load information is less than the second threshold values, the speed that current data can be transmitted is from V
1Be increased to V
3, and V
3=110%V
1If, when the speed of data transmission is V
3The time server load information still less than the second threshold values, then continue to improve the speed of data transmission.
The method of the control data transmission that present embodiment provides, by when Hadoop and relevant database carry out data transmission, connector obtains the load information of the server at relevant database place, then according to the load information control data transmission, reduced the impact that connector causes other business of relevant database when data transmission.
Embodiment three
Referring to Fig. 8, present embodiment provides a kind of method of control data transmission, and described method comprises:
Step 301 ~ 303 are identical with step 201 ~ 203;
Step 304: connector is surveyed the load information of the server that obtains the relevant database place;
Load information can comprise following at least a: the central processing unit load information of server, input and output load information and current EMS memory occupation information, when if the load information of server comprises multiple load information, can be with a load information as current server of currency maximum in the load information of all kinds, for example, the load information of server comprises following three kinds: the central processing unit load information of server, input and output load information and current EMS memory occupation information, and the currency of three kinds of load informations is respectively: ten Percent, 20 percent and 30 percent, then the load information of current server is 30 percent.Connector can periodically be surveyed the load information that obtains server.
Referring to Fig. 9, connector at first obtains the relational data library information, check character and the type of relational data database data, then submit job is to Hadoop, Hadoop carries out data transmission by carrying out a plurality of Map tasks, each Map task is responsible for the transmission of a database table (the perhaps part of a database table), connector is surveyed the load information of the server that obtains the relevant database place, then controls the control that each Map task realizes data transmission according to load information.
Step 305: connector judges according to load information whether described server is overload state, if then execution in step 306, otherwise, execution in step 307;
Wherein, connector adopts following concrete mode to judge whether server is overload state according to load information:
Connector compares load information and the first threshold of presetting;
If load information, determines then that server has reached overload state more than or equal to first threshold;
If load information, determines then that server does not reach overload state less than first threshold.
The user can specify arbitrarily numerical value to the first threshold values.For example, the user can be made as 90% with the first threshold values, if the load information of server, determines that server has reached overload state more than or equal to 90%; If load information, determines then that server does not reach overload state less than 90%.
Step 306: stop data transmission, flow process finishes;
Step 307: continue data transmission, flow process finishes;
Wherein, step 307 can also comprise:
Connector compares load information and the Second Threshold of presetting, if load information more than or equal to the second threshold values, reduces the speed of data transmission, if load information less than the second threshold values, improves the speed of data transmission.
Then the different load informations constantly of record server carry out statistical study to all different load informations constantly that record obtains, and obtain the second threshold values according to the result of statistical study.In addition, the second threshold values also can be specified by the user, and the user can arrange concrete numerical value to the second threshold values according to actual conditions.
For example, the speed of current data transmission is V
1, load information is compared with the second threshold values, if load information greater than the second threshold values, can be with the speed of current data transmission from V
1Be down to V
2, and V
2=90%V
1If, when the speed of data transmission is V
2The time server load information still more than or equal to the second threshold values, then continue to reduce the speed of data transmission; If load information is less than the second threshold values, the speed that current data can be transmitted is from V
1Be increased to V
3, and V
3=110%V
1If, when the speed of data transmission is V
3The time server load information still less than the second threshold values, then continue to improve the speed of data transmission.
The method of the control data transmission that present embodiment provides, by when Hadoop and relevant database carry out data transmission, connector obtains the load information of the server at relevant database place, then according to the load information control data transmission, reduced the impact that connector causes other business of relevant database when data transmission.
Embodiment four
Referring to Figure 10, the embodiment of the invention provides a kind of connector, and this connector comprises: acquisition module 401 and control module 402;
Wherein, acquisition module 401 comprises:
Receiving element is used for receiving the load information that the monitor agent on the server sends;
Perhaps, acquisition module 401 comprises:
Probe unit is used for surveying the load information that obtains server.
Wherein, referring to Figure 11, control module 402 comprises:
Judging unit 4021 is used for judging according to load information whether server is overload state;
Wherein, judging unit 4021 is used for:
Load information and the first threshold of presetting are compared;
If load information, determines then that server has reached overload state more than or equal to first threshold;
If load information, determines then that server does not reach overload state less than first threshold.
Wherein, judging unit 4021 also is used for:
After server does not reach overload state, load information and the Second Threshold of presetting are compared;
If load information more than or equal to the second threshold values, then reduces the speed of data transmission;
If load information less than the second threshold values, then improves the speed of data transmission.
The connector that present embodiment provides, by when Hadoop and relevant database carry out data transmission, connector obtains the load information of the server at relevant database place, then according to the load information control data transmission, reduced the impact that connector causes other business of relevant database when data transmission.
The invention described above embodiment sequence number does not represent the quality of embodiment just to description.
The all or part of step that one of ordinary skill in the art will appreciate that realization above-described embodiment can be finished by hardware, also can come the relevant hardware of instruction to finish by program, described program can be stored in a kind of computer-readable recording medium, the above-mentioned storage medium of mentioning can be ROM (read-only memory), disk or CD etc.
The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of doing, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (12)
1. the method for a control data transmission is characterized in that, described method comprises:
When Hadoop and relevant database carried out data transmission, connector obtained the load information of the server at described relevant database place;
Described connector is controlled described data transmission according to described load information.
2. method according to claim 1 is characterized in that, connector obtains the load information of the server at described relevant database place, comprising:
Described connector receives the load information that the monitor agent on the described server sends;
Perhaps, described connector is surveyed the load information that obtains described server.
3. method according to claim 1 is characterized in that, described connector is controlled described data transmission according to described load information, comprising:
Described connector judges according to described load information whether described server is overload state;
If so, then stop described data transmission; Otherwise, continue described data transmission.
4. method according to claim 3 is characterized in that, described connector judges that according to described load information whether described server is overload state, comprising:
Described connector compares described load information and the first threshold of presetting;
If described load information, determines then that described server has reached overload state more than or equal to described first threshold;
If described load information, determines then that described server does not reach overload state less than described first threshold.
5. method according to claim 4 is characterized in that, determines that described server does not reach after the overload state, also comprises:
Described connector compares described load information and the Second Threshold of presetting;
If described load information more than or equal to described the second threshold values, then reduces the speed of described data transmission;
If described load information less than described the second threshold values, then improves the speed of described data transmission.
6. the described method of any one in 5 according to claim 1 is characterized in that described load information comprises following at least a: the central processing unit load information of described server, input and output load information and current EMS memory occupation information.
7. a connector is characterized in that, described connector comprises: acquisition module and control module;
Described acquisition module is used for obtaining the load information of the server at described relevant database place when Hadoop and relevant database carry out data transmission;
Described control module is used for controlling described data transmission according to described load information.
8. connector according to claim 7 is characterized in that, described acquisition module comprises:
Receiving element is used for receiving the load information that the monitor agent on the described server sends;
Perhaps, probe unit is used for surveying the load information that obtains described server.
9. connector according to claim 7 is characterized in that, described control module comprises:
Judging unit is used for judging according to described load information whether described server is overload state;
Performance element is yes if be used for the result of described judging unit, then stops described data transmission; Otherwise, continue described data transmission.
10. connector according to claim 9 is characterized in that, described judging unit is used for:
Described load information and the first threshold of presetting are compared;
If described load information, determines then that described server has reached overload state more than or equal to described first threshold;
If described load information, determines then that described server does not reach overload state less than described first threshold.
11. connector according to claim 10 is characterized in that, described judging unit also is used for:
After described server does not reach overload state, described load information and the Second Threshold of presetting are compared;
If described load information more than or equal to described the second threshold values, then reduces the speed of described data transmission;
If described load information less than described the second threshold values, then improves the speed of described data transmission.
12. the described connector of any one in 11 is characterized in that described load information comprises following at least a: the central processing unit load information of described server, input and output load information and current EMS memory occupation information according to claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210289213.1A CN102855297B (en) | 2012-08-14 | 2012-08-14 | A kind of method of control data transmission and connector |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210289213.1A CN102855297B (en) | 2012-08-14 | 2012-08-14 | A kind of method of control data transmission and connector |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102855297A true CN102855297A (en) | 2013-01-02 |
| CN102855297B CN102855297B (en) | 2016-04-06 |
Family
ID=47401885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210289213.1A Expired - Fee Related CN102855297B (en) | 2012-08-14 | 2012-08-14 | A kind of method of control data transmission and connector |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102855297B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103176843A (en) * | 2013-03-20 | 2013-06-26 | 百度在线网络技术（北京）有限公司 | File migration method and file migration equipment of Map Reduce distributed system |
| CN106681808A (en) * | 2016-12-01 | 2017-05-17 | 北京奇虎科技有限公司 | Task scheduling method and device |
| CN107145585A (en) * | 2017-05-10 | 2017-09-08 | 温州市鹿城区中津先进科技研究院 | The automated import of data method and system of Hadoop data warehouses |
| CN109189847A (en) * | 2018-09-11 | 2019-01-11 | 国网山东省电力公司莱芜供电公司 | A kind of distribution transforming decreasing loss detection prompt system and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150512A (en) * | 2007-10-23 | 2008-03-26 | 中兴通讯股份有限公司 | Method for load balance of communication link |
| CN101527659A (en) * | 2009-04-15 | 2009-09-09 | 腾讯科技（深圳）有限公司 | Method and system for monitoring data transmission, and network transmitting device |
| US20100162230A1 (en) * | 2008-12-24 | 2010-06-24 | Yahoo! Inc. | Distributed computing system for large-scale data handling |
| CN101800731A (en) * | 2009-02-06 | 2010-08-11 | 株式会社日立制作所 | Network transmission management server, network transmission management method and network transmission system |
-
2012
- 2012-08-14 CN CN201210289213.1A patent/CN102855297B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150512A (en) * | 2007-10-23 | 2008-03-26 | 中兴通讯股份有限公司 | Method for load balance of communication link |
| US20100162230A1 (en) * | 2008-12-24 | 2010-06-24 | Yahoo! Inc. | Distributed computing system for large-scale data handling |
| CN101800731A (en) * | 2009-02-06 | 2010-08-11 | 株式会社日立制作所 | Network transmission management server, network transmission management method and network transmission system |
| CN101527659A (en) * | 2009-04-15 | 2009-09-09 | 腾讯科技（深圳）有限公司 | Method and system for monitoring data transmission, and network transmitting device |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103176843A (en) * | 2013-03-20 | 2013-06-26 | 百度在线网络技术（北京）有限公司 | File migration method and file migration equipment of Map Reduce distributed system |
| CN103176843B (en) * | 2013-03-20 | 2018-12-14 | 百度在线网络技术（北京）有限公司 | The file migration method and apparatus of MapReduce distributed system |
| CN106681808A (en) * | 2016-12-01 | 2017-05-17 | 北京奇虎科技有限公司 | Task scheduling method and device |
| CN107145585A (en) * | 2017-05-10 | 2017-09-08 | 温州市鹿城区中津先进科技研究院 | The automated import of data method and system of Hadoop data warehouses |
| CN109189847A (en) * | 2018-09-11 | 2019-01-11 | 国网山东省电力公司莱芜供电公司 | A kind of distribution transforming decreasing loss detection prompt system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102855297B (en) | 2016-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107506451B (en) | Abnormal information monitoring method and device for data interaction | |
| CN110908997A (en) | Data blood margin construction method and device, server and readable storage medium | |
| CN109189782A (en) | A kind of indexing means in block chain commodity transaction inquiry | |
| CN104090901A (en) | Method, device and server for processing data | |
| CN104572122A (en) | Software application data generating device and method | |
| CN104182405A (en) | Method and device for connection query | |
| CN104090889A (en) | Method and system for data processing | |
| CN103064933A (en) | Data query method and system | |
| CN108021809A (en) | A kind of data processing method and system | |
| CN103927314B (en) | A kind of method and apparatus of batch data processing | |
| CN106294695A (en) | A kind of implementation method towards the biggest data search engine | |
| CN102929961A (en) | Data processing method and device thereof based on building quick data staging channel | |
| CN103034735A (en) | Big data distributed file export method | |
| CN108132868A (en) | A kind of data monitoring method, device, computing device and storage medium | |
| CN103455335A (en) | Multilevel classification Web implementation method | |
| CN109740129B (en) | Report generation method, device and equipment based on blockchain and readable storage medium | |
| CN103699557A (en) | Report processing method and report processing system | |
| CN105683941A (en) | Regulating enterprise database warehouse resource usage | |
| CN102855297A (en) | Method for controlling data transmission, and connector | |
| CN114218218A (en) | Data processing method, device and equipment based on data warehouse and storage medium | |
| CN107247763A (en) | Business datum statistical method, device, system, storage medium and electronic equipment | |
| CN103902256A (en) | Interface generating system and method | |
| CN104090896B (en) | A kind of methods, devices and systems that import data | |
| CN107609172B (en) | Cross-system multi-dimensional data retrieval processing method and device | |
| CN103345527A (en) | Intelligent data statistical system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20180123 Address after: 100176 Beijing Beijing economic and Technological Development Zone Culture Park, No. 6, courtyard No. 30, No. 18, 1803 Patentee after: Beijing Yunhu Times Technology Co., Ltd. Address before: 100020 room 22A01 room C, room No. six, Wantong center, Chaoyang District, Beijing Patentee before: Beijing Grandison & JM Information Technology Co., Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160406 Termination date: 20200814 |