CN100538646C - A kind of method and apparatus of in distributed system, carrying out the SQL script file - Google Patents
A kind of method and apparatus of in distributed system, carrying out the SQL script file Download PDFInfo
- Publication number
- CN100538646C CN100538646C CNB2007101404657A CN200710140465A CN100538646C CN 100538646 C CN100538646 C CN 100538646C CN B2007101404657 A CNB2007101404657 A CN B2007101404657A CN 200710140465 A CN200710140465 A CN 200710140465A CN 100538646 C CN100538646 C CN 100538646C
- Authority
- CN
- China
- Prior art keywords
- buffer zone
- thread
- described buffer
- module
- script
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention provides a kind of method of in distributed system, carrying out the SQL script file, comprising: client terminal start-up first thread and second thread; Described first thread and the operation of second thread parallel; First thread is used for the SQL script file is cut into a plurality of script block and successively each script block is put into the buffer zone that presets; If preset buffer zone is full, then suspend first thread, continue to carry out first thread when preset buffer zone has the space by the time again; Second thread is used for finishing following steps: read a script block of described buffer zone in regular turn, and discharge the respective stored space; Transmit this script block content to server end; After receiving the execution return message of server end, read the next script block in the described buffer zone in regular turn, repeat abovementioned steps, be finished until this SQL script file.The present invention can on average share each data processing point to the executory pressure of file, makes full use of the resource of each several part, thereby improves whole efficient greatly.
Description
Technical field
The present invention relates to data in-line processing field, particularly relate to a kind of method and apparatus of in distributed system, carrying out the SQL script file.
Background technology
Since computing machine comes out, just begin store data, the storage organization of data is managed with regard to one of the most basic problem that has become the required solution of computing machine.Initial data are fought separately, and deposit with document form, different application, different files and identical data have caused the waste of resource and manpower.Increasing along with the Computer Storage data, the problem that data repeat is more and more outstanding, so people just expect data centralization storage, unified management are formed database technology like this with regard to developing into data base management system (DBMS).The research and development of database technology has become the key areas that present information society has powerful vitality.In present most of Database Systems, the application of SQL (Structured Query Language Structured Query Language (SQL)) script is very extensive, the SQL script generally is exactly a kind of " .sq1 " file of writing according to the SQL form, and the SQL script can be carried out by the SQL query analyzer usually.
Most Database Systems all operate in the distributed system in the prior art, and promptly database leaves server end in, and the user can be by the database information of client query and editing server end, and this process just relates to the execution of SQL script file.Again for example, when the function of newly-increased or modified module, sometimes needing increases or revises the structure of background data base or the data in the tables of data, at this moment needs to go to realize to increase newly or requirement during the modified module function to meet with dynamic SQL script file.Especially, when a subsystem to revise or newly-increased function ratio more for a long time, may corresponding SQL script command also can be a lot, at this time Dui Ying SQL script file also can be very big.
In the prior art, general, carrying out the SQL script file all is to adopt all SQL scripts that once all read in the file to be put in the internal memory, then the data in the internal memory is sent to sql server and carries out.For general little SQL script file, this process does not have very big problem basically, but then there is bigger problem for bigger SQL script file, mainly shows as excessively taking of memory source and public resource, and then cause the performance of server end to reduce.In addition, because client is being waited for the execution of sql server end and returned, and the execution of server end is consuming time longer always, the internal memory that then causes client to take can not discharge always, be in the state of on-hook always, bring for other certain operations of client and seriously influence.
In a word, need the urgent technical matters that solves of those skilled in the art to be exactly: the resource when how to reduce big data quantity SQL script file and carrying out in distributed system excessively takies.
Summary of the invention
Technical matters to be solved by this invention provides a kind of method and apparatus of carrying out the SQL script file in distributed system, the SQL script file piecemeal of big data quantity can be sent and carries out, minimizing takies client and server end resource, improves and carries out efficient.
In order to address the above problem, the invention discloses a kind of method of in distributed system, carrying out the SQL script file, specifically comprise:
Client terminal start-up first thread and second thread; Described first thread and the operation of second thread parallel;
Described first thread is used for the SQL script file is cut into a plurality of script block and successively each script block is put into the buffer zone that presets; If preset buffer zone is full, then suspend first thread, continue to carry out first thread again when preset buffer zone has the space by the time, up to all the elements of this SQL script file are all deposited to described buffer zone; Wherein, described buffer zone is set according to the concrete size of this SQL script file and the situation of client hardware resource;
Described second thread is used for finishing following steps: read a script block of described buffer zone in regular turn, and discharge the respective stored space; Transmit this script block content to server end; After receiving the execution return message of server end, read the next script block in the described buffer zone in regular turn, repeat abovementioned steps, be finished until this SQL script file.
Preferably, the SQL script file according to execution sequence, is carried out cutting according to line number.
Preferably, can adopt the mode of first in first out, each script block be formed formation put into described buffer zone; Perhaps, also can carry out putting into described buffer zone after the serial number at each script block.
Preferably, first thread knows by following steps whether described buffer zone has the space: second thread is notified first thread after discharging respective stored space in the described buffer zone; Perhaps, the periodic variation that detects the storage space of described buffer zone of first thread.
Further, described method can also comprise: when second thread reads script block content in the described buffer zone, if buffer zone is empty, then wait for.
According to another embodiment of the present invention, a kind of device of carrying out the SQL script file in distributed system is disclosed, this device is positioned at client, specifically comprises:
First module is used for the SQL script file is cut into a plurality of script block and successively each script block is put into the buffer zone that presets; If preset buffer zone is full, then suspend the execution of this module, continue to move this module again when preset buffer zone has the space by the time, up to all the elements of this SQL script file are all deposited to described buffer zone; Wherein, described buffer zone is set according to the concrete size of this SQL script file and the situation of client hardware resource;
And, comprising second module of reading submodule and communicator module, described reading submodule is used for reading in regular turn a script block of described buffer zone, and discharges the respective stored space; Described communicator module is used to transmit this script block content to server end, and the execution return message that receives server end; Receive the execution return message of server end when described communicator module after, notify described reading submodule to read next script block in the described buffer zone in regular turn; Above-mentioned two submodules rerun, and are finished until this SQL script file;
Described first module and the second module parallel running.
Preferably, the SQL script file according to execution sequence, is carried out cutting according to line number.
Preferably, can adopt the mode of first in first out, each script block be formed formation put into described buffer zone; Perhaps, also can carry out putting into described buffer zone after the serial number at each script block.
Preferably, described device can also comprise: be arranged in the notice submodule of second module, be used for after second module discharges the respective stored space of described buffer zone, notify the described buffer zone of first module Free up Memory; Perhaps, be arranged in the detection sub-module of first module, be used for periodically detecting the variation of the storage space of described buffer zone, whether have the space to know described buffer zone.
Preferably, when described reading submodule reads script block content in the described buffer zone,, then wait for if buffer zone is empty.
Compared with prior art, the present invention has the following advantages:
Because adopting for pending SQL script file, the present invention divide the mode of block cache to operate, do not need the disposable whole taking-ups of the data of this SQL script file, so can save the memory headroom of client, and the buffer zone that is used for depositing each script block can be distributed according to the client calculated performance by the user and gets final product, generally all belong to acceptable scope, can excessively not take the memory source of client.
Secondly, because piecemeal reads, piecemeal is carried out, the execution of each script block can not take too much memory source of server and public resource, can not produce choking phenomenon, so drop to minimum to the influence of server performance.
Moreover because after server end executes a script block, second thread is got back to client just can directly read next script block in the core buffer, thereby mutual cost that can less greatly input and output (I/O) operation improves execution efficient.
In a word, first thread and second thread that the present invention creates, parallel running and cooperatively interacting, improve load balancing degrees, can hang down the task of the SQL script executing of finishing big file internal memory, rapidly and efficiently, the executory pressure of file is on average shared each data processing point, make full use of the resource of each several part, thereby improve whole efficient greatly.
Description of drawings
Fig. 1 is a kind of flow chart of steps of carrying out the method embodiment of SQL script file in distributed system of the present invention;
Fig. 2 is a kind of structured flowchart of carrying out the device embodiment of SQL script file in distributed system of the present invention.
Embodiment
For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.
At first, the inventor has carried out deep analysis at prior art, and is as follows:
(1), client is owing to will store the SQL script file that reads out temporarily, because file is bigger, so need the very big internal memory of application be used for preserving file; Cause excessively taking of client memory source, influence other data handling procedures of client.
(2), because SQL script file data volume is bigger, so cause sql server to bring out existing clogged conditions possibly, can influence the performance of the each side of sql server when serious.At first, if the script that sql server is handled is many more, it is a lot of that the internal memory that takies also can corresponding increase; And, service end can produce a large amount of journal files and wait each status information to write down its operation when carrying out large batch of SQL script, because so the very big journal file that produces of the SQL script file of carrying out also is very big, so can excessively take the memory source of server end.Secondly since need be when carrying out the SQL script log, then must carry out the operation of a large amount of input and output, this process can seriously cause the reduction of server end performance.Moreover, because SQL script file data volume is bigger, associated table and some public resources will be a lot of in the affairs of server end at this SQL script file foundation, these affairs are not finished, then its shared resource just can't be released, can influence other data handling procedures of server end to a great extent to the calling of resource, stop up thereby produce.
At the analysis of foregoing invention people to the technical matters causa essendi of prior art, following a plurality of embodiment have been proposed, solve the problem that resource is excessively taken.
Embodiment proposed by the invention can describe in the general context of the computer executable instructions of being carried out by computing machine, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract, program, object, assembly, data structure or the like.Also can in distributed computing environment, put into practice the present invention, in these distributed computing environment, by by communication network connected teleprocessing equipment execute the task.In distributed computing environment, program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.
With reference to Fig. 1, show a kind of method embodiment that in distributed system, carries out the SQL script file of the present invention, specifically can may further comprise the steps:
Step 101, client terminal start-up first thread and second thread; Described first thread and the operation of second thread parallel.
The present invention is primarily aimed at the improvement of the implementation of the bigger SQL script file of data volume, some ERP (Enterprise Resources Planning, Enterprise Resources Planning) needs to carry out the SQL script file about 10M in, and so huge script file can't be carried out in the SQL query analyzer, open to carry out probably with program in a conventional manner and need 30 minutes, and client-side program is in the state of hang-up always, can not carry out other operation, influence user's use to a great extent.And after adopting the processing mode that the present invention divides block cache, reads and carry out, carry out a same script file, and it is doubly a lot of that speed has improved, and only needs just can finish in about 15 seconds, and certain, the complete later result of program is the same.
The SQL script file what reach for concrete data volume need adopt the present invention, does not need to be limited at this, because along with the difference of client and server end hardware resource, demand can be different.Can judge whether to need to adopt flow performing SQL script file of the present invention by computer self in practice, certainly, also can get final product by technician's manual triggers.After obtaining trigger message, first thread and second thread will be created and start to client.
Step 102, described first thread are used for the SQL script file is cut into a plurality of script block and successively each script block is put into the buffer zone that presets; If preset buffer zone is full, then suspend first thread, continue to carry out first thread when preset buffer zone has the space by the time again.Preferably, can be by the periodic variation that detects the storage space of described buffer zone of first thread, thereby know when described buffer zone has had idle space, to continue carrying out first thread, up to all the elements of this SQL script file are all deposited to described buffer zone.
Described buffer zone can be set as required by computing machine self or user, for example, sets according to the concrete size of this SQL script file and the situation of client hardware resource.
In one embodiment of the invention, step 102 can be divided into the cutting substep and deposit substep.For the cutting substep with deposit relation between the substep, can for: earlier this SQL script file cutting is finished, carries out again and deposit substep,, then suspend and deposit substep if described buffer zone is full.Certainly, cutting substep and deposit relation between the substep, also can for: earlier this SQL script file cutting is obtained a script block, then this script block is deposited to buffer zone, again this SQL script file cutting is obtained next script block, if described buffer zone is full, then suspend the cutting substep.
Does so how concrete this carry out cutting at this SQL script file? in fact, only otherwise any slit mode that influences script executing all is feasible, for example, carry out cutting according to the angle of logic module; Perhaps, carry out cutting or the like according to some the specific markers symbol in the script.In a preferred embodiment of the invention, employing according to execution sequence, is carried out cutting according to line number to the SQL script file; This is a fairly simple slit mode, and generally can not influence execution.As for specifically carrying out cutting according to how many row, then as aforementioned, need determine according to the situation of the situation that specifically takies resource of this SQL script file and client, server end hardware resource, can't provide exact figures at this.
Further, in order to guarantee the correct execution of script, require second thread send to server end carry out the execution script piece must be identical with the order in former SQL script file, in one embodiment of the invention, can adopt arbitrary in the following dual mode, for example, adopt the mode of first in first out, each script block is formed formation puts into described buffer zone, when then second thread reads also the order according to formation read, can not upset the order of script block; Perhaps, first thread carries out putting into described buffer zone after the serial number at each script block.
Step 103, described second thread are used for finishing following steps: read a piece of described buffer zone in regular turn, and discharge the respective stored space; Transmit this piece content to server end; After receiving the execution return message of server end, read the next piece in the described buffer zone in regular turn, repeat abovementioned steps, be finished until this SQL script file.
Described execution return message generally can be the feedback information that is finished of server end.Described second thread can read each script block in the buffer zone according to the queue sequence in the buffer zone or according to number order.
In another preferred embodiment of the present invention, second thread is notified first thread after discharging respective stored space in the described buffer zone, whether has idle space so that first thread is in time known described buffer zone.Second thread whenever reads a script block, just discharges the internal memory of a script block in the described buffer zone, thereby first thread can continue cutting and deposit, and so constantly circulation can be finished reading and carrying out of this SQL script file.
Further, in some cases, when second thread reads piece content in the described buffer zone, may have buffer zone is empty situation, for example, the cutting and the progress of depositing of first thread of client are much slower than implementation progress of server end or the like, then second thread suspension can be carried out and waited for operation this moment.By the time after first thread has been deposited script block in described buffer zone, continue to carry out second thread again.
When the first thread cutting and deposited all pieces of this SQL script file, and second thread transmitted after all script block in the described buffer zone, executing the task of whole SQL script file just finished, and just can destroy first thread and second thread after task is finished.Certainly, when also can first thread finishing, destroy first thread earlier, grade in an imperial examination two threads are destroyed second thread after finishing again.
Because when buffer zone is sky, second thread does not know that reason is first thread end of operation, still cause too slowly owing to speed, therefore, preferred, when first threading operation finishes, notify second thread, at this moment, when buffer zone is empty, then the waiting for server end executes after the current script block, just can finish whole flow process.Certainly, second thread one runs into buffer zone for empty, just hangs up and waits for, if the stand-by period is overtime, thinks that then this SQL script file executed finishes, and then finishes whole flow process.
For aforesaid each method embodiment, for simple description, so it all is expressed as a series of combination of actions, but those skilled in the art should know, the present invention is not subjected to the restriction of described sequence of movement, because according to the present invention, some step can adopt other orders or carry out simultaneously.For example, step 102 and 103 is actually and carries out simultaneously.
With reference to Fig. 2, a kind of device of in distributed system, carrying out the SQL script file, this device is positioned at client 200, it is characterized in that, comprises first module 201 and second module 202; Described first module 201 and 202 parallel runnings of second module.
Described first module 201 is used for the SQL script file is cut into a plurality of and successively each piece is put into the buffer zone that presets; If preset buffer zone is full, then suspend the execution of this module, continue to move this module when preset buffer zone has the space by the time again; Preferably, the SQL script file according to execution sequence, is carried out cutting according to line number.Further,, can adopt the mode of first in first out, each piece be formed formation put into described buffer zone in order to guarantee execution sequence; Perhaps, carry out putting into described buffer zone after the serial number at each piece.
Described second module 202 comprises reading submodule 2021 and communicator module 2022, and described reading submodule 2021 is used for reading in regular turn a piece of described buffer zone, and discharges the respective stored space; Described communicator module 2022 is used to transmit this piece content to server end 300, and the execution return message that receives server end 300; Receive the execution return message of server end 300 when described communicator module 2022 after, notify described reading submodule 2021 to read next piece in the described buffer zone in regular turn; Above-mentioned two submodules rerun, and are finished until this SQL script file.Preferably, when described reading submodule reads piece content in the described buffer zone,, then wait for if buffer zone is empty.If wait timeout is perhaps received the notice that first module is finished, then wait server end to execute after the current script block, just can finish the operation of whole device.
Preferably, know in time that for the ease of first module described buffer zone release has idle space, be arranged in the notice submodule of second module, except finishing above-mentioned functions, can also be used for after second module discharges the respective stored space of described buffer zone, notify the described buffer zone of first module Free up Memory.Perhaps, this device embodiment can also comprise the detection sub-module that is arranged in first module, is used for periodically detecting the variation of the storage space of described buffer zone, whether has the space to know described buffer zone.
Particularly, first module is blocked into big file SQL script the script of fritter and puts into user-defined buffer zone, temporarily stopped piecemeal work if user-defined buffer zone has taken, continue toward the content for script of wherein putting into piecemeal behind the buffer zone Free up Memory by the time big file SQL script.So circulation is until the piecemeal of all finishing the SQL script.Second module reads the SQL script of piecemeal from buffer zone, read after the piecemeal content a then space in buffer release district, the piecemeal content that reads and send to SQL Server server end and carry out, wait for complete follow-up next the piecemeal SQL script of getting buffer zone and send to SQL Server server end of resuming studies that returns.So circulation is that sky is then waited for as the content of buffer zone, finishes the script of whole file up to partition, and it is complete that the content of buffer zone is sent to SQL Server server end all.
More than two modules share out the work and help one another, promptly independently be related again.First module file that breaks by all means is the content of little piecemeal and puts into buffer zone that second module then only sends to server from the buffer zone reading of content.Two modules are got in touch by public buffer zone.By top two modules can well balance client and server end resource pressure, thereby on performance and efficient, be greatly improved.
Each embodiment in this instructions all adopts the mode of going forward one by one to describe, and what each embodiment stressed all is and the difference of other embodiment that identical similar part is mutually referring to getting final product between each embodiment.For device embodiment, because it is similar substantially to method embodiment, so description is fairly simple, relevant part gets final product referring to the part explanation of method embodiment.Secondly, those skilled in the art also should know, the embodiment described in the instructions all belongs to preferred embodiment, and related action and module might not be that the present invention is necessary.
More than to a kind of method and apparatus of in distributed system, carrying out the SQL script file provided by the present invention, be described in detail, used specific case herein principle of the present invention and embodiment are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that all can change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (10)
1, a kind of method of carrying out the SQL script file in distributed system is characterized in that, comprising:
Client terminal start-up first thread and second thread; Described first thread and the operation of second thread parallel;
Described first thread is used for the SQL script file is cut into a plurality of script block and successively each script block is put into the buffer zone that presets; If preset buffer zone is full, then suspend first thread, continue to carry out first thread again when preset buffer zone has the space by the time, up to all the elements of this SQL script file are all deposited to described buffer zone; Wherein, described buffer zone is set according to the concrete size of this SQL script file and the situation of client hardware resource;
Described second thread is used to finish following steps:
Read a script block in the described buffer zone in regular turn, and discharge the respective stored space;
Transmit this script block content to server end;
After receiving the execution return message of server end, read the next script block in the described buffer zone in regular turn, repeat abovementioned steps, be finished until this SQL script file.
2, the method for claim 1 is characterized in that, the SQL script file according to execution sequence, is carried out cutting according to line number.
3, the method for claim 1 is characterized in that,
Adopt the mode of first in first out, each script block is formed formation put into described buffer zone;
Perhaps, carry out putting into described buffer zone after the serial number at each script block.
4, the method for claim 1 is characterized in that, first thread knows by following steps whether described buffer zone has the space:
Second thread is notified first thread after discharging respective stored space in the described buffer zone;
Perhaps, the periodic variation that detects the storage space of described buffer zone of first thread.
5, the method for claim 1 is characterized in that, also comprises,
When second thread reads script block content in the described buffer zone,, then wait for if buffer zone is empty.
6, a kind of device of in distributed system, carrying out the SQL script file, this device is positioned at client, it is characterized in that, comprising:
First module is used for the SQL script file is cut into a plurality of script block and successively each script block is put into the buffer zone that presets; If preset buffer zone is full, then suspend the execution of this module, continue to move this module again when preset buffer zone has the space by the time, up to all the elements of this SQL script file are all deposited to described buffer zone; Wherein, described buffer zone is set according to the concrete size of this SQL script file and the situation of client hardware resource;
Second module that comprises reading submodule and communicator module, described reading submodule are used for reading in regular turn a script block of described buffer zone, and discharge the respective stored space; Described communicator module is used to transmit this script block content to server end, and the execution return message that receives server end; Receive the execution return message of server end when described communicator module after, notify described reading submodule to read next script block in the described buffer zone in regular turn; Above-mentioned two submodules rerun, and are finished until this SQL script file;
Described first module and the second module parallel running.
7, device as claimed in claim 6 is characterized in that, the SQL script file according to execution sequence, is carried out cutting according to line number.
8, device as claimed in claim 6 is characterized in that,
Adopt the mode of first in first out, each script block is formed formation put into described buffer zone;
Perhaps, carry out putting into described buffer zone after the serial number at each script block.
9, device as claimed in claim 6 is characterized in that, also comprises:
Be arranged in the notice submodule of second module, be used for after second module discharges the respective stored space of described buffer zone, notify the described buffer zone of first module Free up Memory;
Perhaps, be arranged in the detection sub-module of first module, be used for periodically detecting the variation of the storage space of described buffer zone, whether have the space to know described buffer zone.
10, device as claimed in claim 6 is characterized in that,
When described reading submodule reads script block content in the described buffer zone,, then wait for if buffer zone is empty.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007101404657A CN100538646C (en) | 2007-08-14 | 2007-08-14 | A kind of method and apparatus of in distributed system, carrying out the SQL script file |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007101404657A CN100538646C (en) | 2007-08-14 | 2007-08-14 | A kind of method and apparatus of in distributed system, carrying out the SQL script file |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101093454A CN101093454A (en) | 2007-12-26 |
| CN100538646C true CN100538646C (en) | 2009-09-09 |
Family
ID=38991729
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2007101404657A Active CN100538646C (en) | 2007-08-14 | 2007-08-14 | A kind of method and apparatus of in distributed system, carrying out the SQL script file |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100538646C (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102479225B (en) * | 2010-11-26 | 2014-05-07 | 中国移动通信集团四川有限公司 | Distributed data analyzing and processing method and system |
| CN102567024A (en) * | 2011-12-28 | 2012-07-11 | 畅捷通信息技术股份有限公司 | Script executing system and method |
| CN105843819B (en) * | 2015-01-15 | 2020-07-10 | 中国移动通信集团河南有限公司 | Data export method and device |
| KR101727272B1 (en) * | 2016-01-19 | 2017-04-17 | 쉬프트정보통신 주식회사 | Method for divisionally transmitting and processing data between client and server in JavaScript language based asynchronous communication environment |
| CN110121704B (en) * | 2016-12-30 | 2021-09-21 | 华为技术有限公司 | Execution method, device and system for stored procedure |
| CN109597810B (en) * | 2018-11-21 | 2021-11-09 | 金色熊猫有限公司 | Task segmentation method, device, medium and electronic equipment |
| CN109831485A (en) * | 2018-12-29 | 2019-05-31 | 芜湖哈特机器人产业技术研究院有限公司 | A kind of data communication and analytic method of laser radar |
| CN110362613A (en) * | 2019-07-22 | 2019-10-22 | 中国联合网络通信集团有限公司 | Data processing method and equipment |
| CN111191794B (en) * | 2019-12-29 | 2023-03-14 | 广东浪潮大数据研究有限公司 | Training task processing method, device and equipment and readable storage medium |
| CN111177106B (en) * | 2019-12-30 | 2023-10-17 | 嘉联支付有限公司 | Distributed data computing system and method |
-
2007
- 2007-08-14 CN CNB2007101404657A patent/CN100538646C/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN101093454A (en) | 2007-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100538646C (en) | A kind of method and apparatus of in distributed system, carrying out the SQL script file | |
| US20230385262A1 (en) | System And Method For Large-Scale Data Processing Using An Application-Independent Framework | |
| Marcu et al. | Spark versus flink: Understanding performance in big data analytics frameworks | |
| CN102521712B (en) | A kind of process instance data processing method and device | |
| US20200057672A1 (en) | Dynamic tree determination for data processing | |
| EP2503733B1 (en) | Data collecting method, data collecting apparatus and network management device | |
| US8738645B1 (en) | Parallel processing framework | |
| WO2019109854A1 (en) | Data processing method and device for distributed database, storage medium, and electronic device | |
| CN111324606B (en) | Data slicing method and device | |
| CN109885642B (en) | Hierarchical storage method and device for full-text retrieval | |
| WO2019223599A1 (en) | Data acquisition system and method, node device and storage medium | |
| CN102360314A (en) | System and method for managing resources of data center | |
| US8407713B2 (en) | Infrastructure of data summarization including light programs and helper steps | |
| CN115373835A (en) | Task resource adjusting method and device for Flink cluster and electronic equipment | |
| CN114968953A (en) | Log storage and retrieval method, system, terminal equipment and medium | |
| CN111309805B (en) | Data reading and writing method and device for database | |
| CN112579287A (en) | Cloud arrangement system and method based on read-write separation and automatic expansion | |
| CN113157411B (en) | Celery-based reliable configurable task system and device | |
| US20230325242A1 (en) | Fast shutdown of large scale-up processes | |
| CN106815318B (en) | Clustering method and system for time sequence database | |
| CN116302574B (en) | Concurrent processing method based on MapReduce | |
| US20170235602A1 (en) | Resource Scheduling Method and Related Apparatus | |
| CN112199432A (en) | High-performance data ETL device based on distribution and control method | |
| CN113485812B (en) | Partition parallel processing method and system based on large-data-volume task | |
| CN112100186B (en) | Data processing method and device based on distributed system and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |