CN105468445B - WEB-based Spark application program scheduling method and system - Google Patents
WEB-based Spark application program scheduling method and system Download PDFInfo
- Publication number
- CN105468445B CN105468445B CN201510805815.1A CN201510805815A CN105468445B CN 105468445 B CN105468445 B CN 105468445B CN 201510805815 A CN201510805815 A CN 201510805815A CN 105468445 B CN105468445 B CN 105468445B
- Authority
- CN
- China
- Prior art keywords
- application program
- spark application
- spark
- execution
- web
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/323—Visualisation of programs or trace data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Stored Programmes (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a WEB-based Spark application program scheduling method and a system, wherein the method comprises the following steps: A. selecting available Spark application programs from a Spark application program library through WEB, adding the Spark application programs into statistical activity, editing the dependency relationship of each Spark application program, generating a dependency relationship graph and execution sequence information, and storing the dependency relationship graph and the execution sequence information into a scheduling information database; B. and starting statistical activities, sequentially executing the Spark application programs according to the execution sequence information, and recording the states of the Spark application programs. The invention configures the dependency relationship between the Spark application programs through the browser and can display the dependency relationship in a graphical mode. The operation and maintenance personnel can monitor various states of the Spark application program, the operation and maintenance efficiency is greatly improved, and the error probability of the operation and maintenance is reduced.
Description
Technical Field
The invention relates to the field of distributed computing, in particular to a WEB-based Spark application program scheduling method and system.
Background
Spark is an open source cluster computing system based on memory computing. As Spark becomes the top-level open source project of the Apache foundation, more and more people use Spark in big data computing. Spark can operate in multiple modes such as Standalone, Yarn, meso (Spark is a common installation and deployment mode).
DAG (direct acyclic graph) is used inside the Spark to maintain the dependency relationship between jobs inside the Spark application, and the dependency relationship between the Spark applications needs to be maintained in a way that an executor manually records the execution sequence, which is inconvenient to use. And once a certain Application fails to execute, the operation and maintenance personnel easily miss the Spark Application program which needs to be executed together, thereby causing the distortion of the statistical data.
Accordingly, the prior art is yet to be improved and developed.
Disclosure of Invention
In view of the defects of the prior art, the present invention aims to provide a method and a system for scheduling a Spark application based on WEB, and aims to solve the problems that the existing Spark is inconvenient to use and easy to make mistakes in maintenance.
The technical scheme of the invention is as follows:
a WEB-based Spark application program scheduling method comprises the following steps:
A. selecting available Spark application programs from a Spark application program library through WEB, adding the Spark application programs into statistical activity, editing the dependency relationship of each Spark application program, generating execution sequence information according to the dependency relationship, and storing the execution sequence information in a scheduling information database;
B. and starting statistical activities, sequentially executing the Spark application programs according to the execution sequence information, and recording the states of the Spark application programs.
The WEB-based Spark application program scheduling method comprises the following steps: if the spare application program does not exist in the spare application program library, adding the spare application program into the spare application program library by entering a spare application program management function page;
the step A also comprises the following steps: and judging whether the dependency relationship graph has a ring or not, if so, checking the dependency relationship graph not to pass, returning to re-edit the dependency relationship of each Spark application program, and if so, generating execution sequence information.
The WEB-based Spark application program scheduling method includes the following specific steps:
b1, executing a command for starting statistical activity by executing the shell script;
b2, acquiring execution sequence information from the scheduling information database, sequentially generating execution scripts of the Spark application program according to the execution sequence information, and submitting the generated execution scripts to an execution machine for execution;
b3, if the Spark application program is executed successfully, recording the corresponding execution information into the statistical result database, and simultaneously setting the state of the Spark application program as 'finished'; if the execution fails, setting the state of the Spark application program as 'failure';
b4, after all Spark applications are executed, checking whether Spark applications with the state of 'failure' exist in the statistical activity, if so, returning to the step B2 to execute the statistical activity again, and if the execution times of the statistical activity is less than the preset times; if the statistical activity exists and the execution times of the statistical activity are larger than or equal to the preset times, the execution failure of the statistical activity is identified.
In the method for scheduling Spark application programs based on WEB, in the step a, editing the dependency relationship of each Spark application program specifically includes:
each Spark application is assigned a parent Spark application, and the root Spark application does not have a parent Spark application.
The WEB-based Spark application program scheduling method comprises the following steps: and counting the number of times that each Spark application program is referred to and the referred statistical activity information in real time.
A WEB-based Spark application scheduling system, comprising:
the editing module is used for selecting available Spark application programs from the Spark application program library through WEB, adding the Spark application programs into statistical activity, editing the dependency relationship of each Spark application program, generating execution sequence information according to the dependency relationship, and storing the execution sequence information into the scheduling information database;
and the execution module is used for starting the statistical activity, sequentially executing the Spark application programs according to the execution sequence information, and recording the states of the Spark application programs.
The WEB-based Spark application scheduling system, wherein the editing module further comprises:
the adding unit is used for adding the Spark application program into the Spark application program library by entering a Spark application program management function page if the Spark application program is not available in the Spark application program library;
and the checking unit is used for judging whether the dependency graph has a ring or not, if so, the checking fails, returning to re-edit the dependency of each Spark application program, and if so, generating execution sequence information.
The WEB-based Spark application scheduling system comprises an execution module, a scheduling module and a scheduling module, wherein the execution module specifically comprises:
the starting unit is used for executing a command for starting the statistical activity by executing the shell script;
the execution unit is used for acquiring the execution sequence information from the scheduling information database, sequentially generating the execution scripts of the Spark application program according to the execution sequence information, and submitting the generated execution scripts to the execution machine for execution;
a recording unit, configured to record, if the Spark application is successfully executed, corresponding execution information to the statistical result database, and set the state of the Spark application to "complete"; if the execution fails, setting the state of the Spark application program as 'failure';
the checking unit is used for checking whether the Spark application program with the state of failure exists in the statistical activity after all Spark application programs are executed, and returning to the executing unit to execute the statistical activity again if the Spark application program exists and the execution times of the statistical activity is less than the preset times; if the statistical activity exists and the execution times of the statistical activity are larger than or equal to the preset times, the execution failure of the statistical activity is identified.
The WEB-based Spark application scheduling system, wherein the editing module further comprises:
and the specifying unit is used for specifying a parent Spark application program for each Spark application program, and the parent Spark application program does not exist in the root Spark application program.
The WEB-based Spark application scheduling system, wherein the execution module further comprises:
and the counting unit is used for counting the number of times that each Spark application program is quoted and the quoted counting activity information in real time.
Has the advantages that: the invention configures the dependency relationship between Spark applications (Application) through the browser and can be displayed in a graphical mode. The operation and maintenance personnel can monitor various states of the Spark application program, the operation and maintenance efficiency is greatly improved, and the error probability of the operation and maintenance is reduced.
Drawings
Fig. 1 is a flowchart of a preferred embodiment of a method for scheduling a Spark application based on WEB according to the present invention.
Fig. 2 is a flowchart of another embodiment of a method for scheduling a Spark application based on WEB according to the present invention.
Fig. 3 is a detailed flowchart of step S101 in the method shown in fig. 1.
Fig. 4 is a detailed flowchart of step S102 in the method shown in fig. 1.
Fig. 5 is a block diagram illustrating a preferred embodiment of a WEB-based Spark application scheduling system according to the present invention.
Fig. 6 is a block diagram showing a specific structure of an execution module in the system shown in fig. 5.
Detailed Description
The invention provides a WEB-based Spark application program scheduling method and system, and the invention is further described in detail below in order to make the purpose, technical scheme and effect of the invention clearer and more clear. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating a preferred embodiment of a method for scheduling a WEB-based Spark application according to the present invention, and as shown in the figure, the method includes the following steps:
s101, selecting available Spark application programs from a Spark application program library through WEB, adding the Spark application programs into statistical activity, editing the dependency relationship of each Spark application program, generating a dependency relationship graph according to the dependency relationship of each Spark application program, generating execution sequence information, and storing the execution sequence information into a scheduling information database;
and S102, starting statistical activities, sequentially executing the Spark application programs according to the execution sequence information, and recording the states of the Spark application programs.
In the embodiment of the invention, the function which is relatively independent in service is used as a Spark application program, and one statistical activity is composed of a plurality of Spark application programs and is sequentially executed according to the sequence. The Spark Application may also be referred to as Application in the present invention. The dependency relationship is configured through the WEB, the dependency relationship graph is generated, the graph is displayed in a graphical mode, and operation and maintenance personnel can monitor the state of each Spark application program, so that the operation and maintenance efficiency is improved, and the error probability is reduced.
As shown in fig. 2, the whole scheduling method includes the following steps:
a: logging in a WEB-based Spark application program scheduling system through a browser and carrying out related operations.
b: the WEB server 10 stores the information added by the user in a scheduling information database, which stores Spark application programs and information related to statistical activities.
c: the Master server (main server) 20 acquires information of an Application list to be executed from the scheduling information database.
d: the Master server 20 submits the Application to the managed Worker Server.
e: the Worker server (work server) 30 saves the statistical result in the statistical result database.
As shown in fig. 3, the step S101 specifically includes:
s201, logging in a system (a Web-based Spark application scheduling system) through a browser;
s202, entering a statistical activity management function page, adding statistical activity, and filling out the name of the statistical activity and other related information, such as activity description, creation time and the like. In addition to the addition, various operations such as editing, deleting statistical activities, and the like may be performed.
S203, selecting an available Spark Application program from the Spark Application program library and adding the Spark Application program into the statistical activity, if the Spark Application program does not exist in the Spark Application program library, entering a Spark Application program management function page, adding the Spark Application program into the Spark Application program library, and continuing to add the available Application program into the statistical activity. In addition to the addition, various operations such as editing, deleting the Spark application, and the like can be performed.
The contents of the Spark application are shown in table 1, which includes a primary key, an application name, an application description, the number of times of multiplexing, and a creation time.
TABLE 1
The content of the statistical activity is shown in table 2, which includes the primary key, the activity name, the activity description, and the creation time.
TABLE 2
S204, editing the dependency relationship of each Spark application program in the statistical activity, specifically: each Spark application is assigned a parent Spark application, and the root Spark application does not have a parent Spark application. And two Spark applications can only depend on one way and can not depend on the loop.
The content of the dependency relationship of the Spark application program is shown in table 3, which includes a primary key, a parent application id, an application id, a belonging statistical activity id, an application description, a CPU allocation number, an internal memory allocation, an application state, an execution duration, and a creation time.
TABLE 3
S205, after the dependency relationship of each Spark application program is edited, generating a dependency relationship graph according to the dependency relationship of each Spark application program;
s206, checking the dependency graph, and judging whether the dependency graph has a loop, namely the generated dependency graph must be a directed acyclic graph, so as to avoid a dead cycle phenomenon during execution. Specifically, whether a ring exists in the dependency graph is determined by a depth-first traversal algorithm of the graph, and as for the algorithm, reference may be made to the content of the prior art. When the check fails, the dependency relationship of each Spark application needs to be edited again. Or the operation and maintenance personnel can visually check whether the dependency relationship of each Spark application program is correct through the dependency relationship diagram.
And S207, when the verification is passed, generating execution sequence information of each Spark application program, specifically generating the execution sequence information through a breadth-first traversal algorithm of the graph, and then storing the generated execution sequence information in a scheduling information database. Reference may be made to the prior art with respect to breadth first traversal algorithms of graphs. At this point, the entire statistical activity compilation is complete.
Further, as shown in fig. 4, the step S102 specifically includes:
s301, executing a command for starting statistical activity by executing a shell script; such as by executing the shell script of Linux.
S302, acquiring execution sequence information from a scheduling information database, sequentially generating execution scripts of Spark application programs according to the execution sequence information, and submitting the generated execution scripts to an execution machine for execution;
s303, if the Spark application program is executed successfully, recording corresponding execution information to a statistical result database, and meanwhile, setting the state of the Spark application program to be 'FINISHED' (FINISHED); if the execution FAILs, setting the state of the Spark application program as Failure (FAIL);
s304, after all Spark applications are executed, checking whether Spark applications in a failure state exist in the statistical activity, and if the Spark applications exist and the execution times of the statistical activity are less than the preset times, returning to the step S302 to execute the statistical activity again; if the statistical activity exists and the execution times of the statistical activity are larger than or equal to the preset times, the execution failure of the statistical activity is identified. The predetermined number of times may be 3 times, and of course, the predetermined number of times may be adjusted as needed.
Further, the step S102 further includes: and counting the number of times that each Spark application program is referred to and the referred statistical activity information in real time.
In the invention, operation and maintenance personnel can obtain the activity (multiplexing times) of each Spark application program, and can inquire the activity according to the statistical activity or the name of the Spark application program. The liveness may be sorted in ascending or descending order. In addition, the Spark application with the activity of 0 can be prompted regularly, for example, an email of Spark application information with the activity of 0 is sent to the operation and maintenance personnel at 8 am every day.
The invention realizes the method for decoupling the Spark application programs, so that a user can configure the dependency relationship among the Spark application programs more easily, only needs to pay attention to the realization of respective services in the development process, the capability of collaborative development is greatly enhanced, and the maintainability of the system is greatly improved. The operation and maintenance personnel can monitor the number of times each Spark application program is quoted and the statistical activity of the Spark application programs, and the activity of the Spark application programs can be judged by monitoring the number of times the Spark application programs are quoted.
Based on the foregoing method, the present invention further provides a block diagram of a preferred embodiment of a WEB-based Spark application scheduling system, as shown in fig. 5, where the block diagram includes:
the editing module 100 is configured to select an available Spark application from a Spark application library through the WEB, add the selected Spark application to the statistical activity, edit the dependency relationship of each Spark application, generate a dependency relationship graph according to the dependency relationship of each Spark application, generate execution sequence information, and store the execution sequence information in the scheduling information database;
and the execution module 200 is configured to start the statistical activity, sequentially execute each Spark application according to the execution sequence information, and record the state of each Spark application.
Further, the editing module 100 further includes:
the adding unit is used for adding the Spark application program into the Spark application program library by entering a Spark application program management function page if the Spark application program is not available in the Spark application program library;
and the checking unit is used for judging whether the dependency graph has a ring or not, if so, the checking fails, returning to re-edit the dependency of each Spark application program, and if so, generating execution sequence information.
Further, as shown in fig. 6, the execution module 200 specifically includes:
a starting unit 210, configured to execute a command for starting a statistical activity by executing a shell script;
an executing unit 220, configured to obtain execution sequence information from the scheduling information database, sequentially generate execution scripts of the Spark application program according to the execution sequence information, and submit the generated execution scripts to an executing machine for execution;
a recording unit 230, configured to record, if the Spark application is successfully executed, corresponding execution information in the statistical result database, and set the state of the Spark application to "complete"; if the execution fails, setting the state of the Spark application program as 'failure';
a checking unit 240, configured to check whether there is a spare application program in a "failure" state in the statistical activity after all spare application programs are executed, and if there is a spare application program in the statistical activity and the execution frequency of the statistical activity is less than the predetermined frequency, return to the executing unit to execute the statistical activity again; if the statistical activity exists and the execution times of the statistical activity are larger than or equal to the preset times, the execution failure of the statistical activity is identified.
Further, the editing module 100 further includes:
and the specifying unit is used for specifying a parent Spark application program for each Spark application program, and the parent Spark application program does not exist in the root Spark application program.
Further, the executing module 200 further includes:
and the counting unit is used for counting the number of times that each Spark application program is quoted and the quoted counting activity information in real time.
The technical details of the above module unit have been described in the foregoing method, and thus are not described again.
In summary, the invention configures the dependency relationship between the Spark applications (applications) through the browser, and can be graphically displayed. Operation and maintenance personnel can monitor various states of Spark Application programs (Application), operation and maintenance efficiency is greatly improved, and operation and maintenance error probability is reduced.
It is to be understood that the invention is not limited to the examples described above, but that modifications and variations may be effected thereto by those of ordinary skill in the art in light of the foregoing description, and that all such modifications and variations are intended to be within the scope of the invention as defined by the appended claims.
Claims (10)
1. A WEB-based Spark application program scheduling method is characterized by comprising the following steps:
A. selecting available Spark application programs from a Spark application program library through WEB, adding the Spark application programs into statistical activity, editing the dependency relationship of each Spark application program, generating a dependency relationship graph according to the dependency relationship of each Spark application program, generating execution sequence information, and storing the execution sequence information in a scheduling information database;
B. starting statistical activities, sequentially executing the Spark application programs according to the execution sequence information, and recording the states of the Spark application programs;
the step A further comprises the following steps: and if the spare application program does not exist in the spare application program library, entering a spare application program management function page, and adding the spare application program into the spare application program library.
2. The WEB-based Spark application scheduling method according to claim 1, wherein the step a further includes: and judging whether the dependency relationship graph has a ring or not, if so, checking the dependency relationship graph not to pass, returning to re-edit the dependency relationship of each Spark application program, and if so, generating execution sequence information.
3. The WEB-based Spark application scheduling method according to claim 1, wherein the step B specifically comprises:
b1, executing a command for starting statistical activity by executing the shell script;
b2, acquiring execution sequence information from the scheduling information database, sequentially generating execution scripts of the Spark application program according to the execution sequence information, and submitting the generated execution scripts to an execution machine for execution;
b3, if the Spark application program is executed successfully, recording the corresponding execution information into the statistical result database, and simultaneously setting the state of the Spark application program as 'finished'; if the execution fails, setting the state of the Spark application program as 'failure';
b4, after all Spark applications are executed, checking whether Spark applications with the state of 'failure' exist in the statistical activity, if so, returning to the step B2 to execute the statistical activity again, and if the execution times of the statistical activity is less than the preset times; if the statistical activity exists and the execution times of the statistical activity are larger than or equal to the preset times, the execution failure of the statistical activity is identified.
4. The WEB-based Spark application scheduling method according to claim 1, wherein in the step a, editing the dependency relationship of each Spark application specifically includes:
each Spark application is assigned a parent Spark application, and the root Spark application does not have a parent Spark application.
5. The WEB-based Spark application scheduling method according to claim 1, wherein the step B further comprises: and counting the number of times that each Spark application program is referred to and the referred statistical activity information in real time.
6. A WEB-based Spark application scheduling system, comprising:
the editing module is used for selecting available Spark application programs from the Spark application program library through WEB, adding the Spark application programs into statistical activity, editing the dependency relationship of each Spark application program, generating a dependency relationship graph according to the dependency relationship of each Spark application program, generating execution sequence information, and storing the execution sequence information into the scheduling information database;
if the spare application program does not exist in the spare application program library, entering a spare application program management function page, and adding the spare application program into the spare application program library;
and the execution module is used for starting the statistical activity, sequentially executing the Spark application programs according to the execution sequence information, and recording the states of the Spark application programs.
7. The WEB-based Spark application scheduling system of claim 6 wherein said editing module further comprises:
the adding unit is used for adding the Spark application program into the Spark application program library by entering a Spark application program management function page if the Spark application program is not available in the Spark application program library;
and the checking unit is used for judging whether the dependency graph has a ring or not, if so, the checking fails, returning to re-edit the dependency of each Spark application program, and if so, generating execution sequence information.
8. The WEB-based Spark application scheduling system according to claim 6, wherein the execution module specifically comprises:
the starting unit is used for executing a command for starting the statistical activity by executing the shell script;
the execution unit is used for acquiring the execution sequence information from the scheduling information database, sequentially generating the execution scripts of the Spark application program according to the execution sequence information, and submitting the generated execution scripts to the execution machine for execution;
a recording unit, configured to record, if the Spark application is successfully executed, corresponding execution information to the statistical result database, and set the state of the Spark application to "complete"; if the execution fails, setting the state of the Spark application program as 'failure';
the checking unit is used for checking whether the Spark application program with the state of failure exists in the statistical activity after all Spark application programs are executed, and returning to the executing unit to execute the statistical activity again if the Spark application program exists and the execution times of the statistical activity is less than the preset times; if the statistical activity exists and the execution times of the statistical activity are larger than or equal to the preset times, the execution failure of the statistical activity is identified.
9. The WEB-based Spark application scheduling system of claim 6 wherein said editing module further comprises:
and the specifying unit is used for specifying a parent Spark application program for each Spark application program, and the parent Spark application program does not exist in the root Spark application program.
10. The WEB-based Spark application scheduling system of claim 6, wherein the execution module further comprises:
and the counting unit is used for counting the number of times that each Spark application program is quoted and the quoted counting activity information in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510805815.1A CN105468445B (en) | 2015-11-20 | 2015-11-20 | WEB-based Spark application program scheduling method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510805815.1A CN105468445B (en) | 2015-11-20 | 2015-11-20 | WEB-based Spark application program scheduling method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105468445A CN105468445A (en) | 2016-04-06 |
| CN105468445B true CN105468445B (en) | 2020-01-14 |
Family
ID=55606180
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510805815.1A Active CN105468445B (en) | 2015-11-20 | 2015-11-20 | WEB-based Spark application program scheduling method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105468445B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106202556B (en) * | 2016-07-28 | 2019-08-30 | 中国电子科技集团公司第二十八研究所 | A kind of mass text keyword rapid extracting method based on Spark |
| CN107391266B (en) * | 2017-06-01 | 2021-03-30 | 华南理工大学 | Graphical programming multithreading synchronization method |
| CN109284888A (en) * | 2018-06-19 | 2019-01-29 | 杭州数澜科技有限公司 | A kind of method and apparatus for the loop between Detection task dependence |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794239A (en) * | 2010-03-16 | 2010-08-04 | 浙江大学 | Multiprocessor task scheduling management method based on data flow model |
| CN102323945A (en) * | 2011-09-02 | 2012-01-18 | 南京中兴力维软件有限公司 | SQL (Structured Query Language)-based database management method and device |
| CN103080900A (en) * | 2010-09-03 | 2013-05-01 | 西门子公司 | Method for parallelizing automatic control programs and compiler |
| CN103377035A (en) * | 2012-04-12 | 2013-10-30 | 浙江大学 | Pipeline parallelization method for coarse-grained streaming application |
| CN103678505A (en) * | 2013-11-20 | 2014-03-26 | 北京奇虎科技有限公司 | Method and device for running application program in browser and browser |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9286140B2 (en) * | 2008-12-24 | 2016-03-15 | International Business Machines Corporation | Remotely monitoring and scheduling a data integration job |
| CN103631730B (en) * | 2013-11-01 | 2016-04-27 | 深圳清华大学研究院 | The cache optimization method that internal memory calculates |
-
2015
- 2015-11-20 CN CN201510805815.1A patent/CN105468445B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794239A (en) * | 2010-03-16 | 2010-08-04 | 浙江大学 | Multiprocessor task scheduling management method based on data flow model |
| CN103080900A (en) * | 2010-09-03 | 2013-05-01 | 西门子公司 | Method for parallelizing automatic control programs and compiler |
| CN102323945A (en) * | 2011-09-02 | 2012-01-18 | 南京中兴力维软件有限公司 | SQL (Structured Query Language)-based database management method and device |
| CN103377035A (en) * | 2012-04-12 | 2013-10-30 | 浙江大学 | Pipeline parallelization method for coarse-grained streaming application |
| CN103678505A (en) * | 2013-11-20 | 2014-03-26 | 北京奇虎科技有限公司 | Method and device for running application program in browser and browser |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105468445A (en) | 2016-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10296305B2 (en) | Method and device for the automated production and provision of at least one software application | |
| CN110069572A (en) | HIVE method for scheduling task, device, equipment and storage medium based on big data platform | |
| US20090287643A1 (en) | Context based script generation | |
| JP6045134B2 (en) | Parallel workload simulation for application performance testing | |
| US7827273B2 (en) | Machine cluster topology representation for automated testing | |
| CN104679717A (en) | Method and management system of elastic cluster deployment | |
| US11138097B2 (en) | Automated web testing framework for generating and maintaining test scripts | |
| CN108920139B (en) | Program generation method, device and system, electronic equipment and storage medium | |
| CN105468445B (en) | WEB-based Spark application program scheduling method and system | |
| US11416512B2 (en) | Systems and methods for facilitating data transformation | |
| CN112835924A (en) | Real-time computing task processing method, device, equipment and storage medium | |
| CN114741375A (en) | Rapid and automatic data migration system and method for multi-source heterogeneous database | |
| CN106600226A (en) | Method and device used for optimizing flow management system | |
| US11531528B2 (en) | Systems and methods for non-disruptive continuous software delivery | |
| CN106547861A (en) | A kind of method and device of the data base of intelligent management machine node | |
| CN110502242A (en) | Code automatic generation method, device, computer equipment and storage medium | |
| CN105117329A (en) | Application automatic online system and method | |
| CN113687927A (en) | Method, device, equipment and storage medium for scheduling and configuring flash tasks | |
| CN114168287A (en) | Task scheduling method and device, readable storage medium and electronic equipment | |
| CN111625330A (en) | Cross-thread task processing method and device, server and storage medium | |
| CN116627437A (en) | Deployment method and device of Airflow service, storage medium and computer equipment | |
| CN112256978B (en) | Data processing method, device and medium based on data model | |
| CN114489999A (en) | Method and device for processing pipeline task, processor and electronic equipment | |
| Kim et al. | Management of software test using case tool | |
| Uddagiri et al. | Improving the quality of requirements in middleware requirements specifications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |