CN111199079A - Simulation task scheduling method and device - Google Patents
Simulation task scheduling method and device Download PDFInfo
- Publication number
- CN111199079A CN111199079A CN201811277421.3A CN201811277421A CN111199079A CN 111199079 A CN111199079 A CN 111199079A CN 201811277421 A CN201811277421 A CN 201811277421A CN 111199079 A CN111199079 A CN 111199079A
- Authority
- CN
- China
- Prior art keywords
- simulation
- task
- operator
- simulation task
- calling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 268
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000000977 initiatory effect Effects 0.000 claims abstract description 41
- 238000012546 transfer Methods 0.000 claims description 18
- 238000012163 sequencing technique Methods 0.000 claims description 10
- 230000003993 interaction Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000000638 solvent extraction Methods 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 abstract description 11
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/133—Protocols for remote procedure calls [RPC]
Abstract
The embodiment of the invention provides a method and a device for scheduling a simulation task. The method comprises the following steps: receiving each simulation task initiation request aiming at a plurality of simulation scenes; calling each simulation operator by adopting a pipeline scheduling mode to initiate a request aiming at each simulation task to generate each simulation task; and sending each simulation task to a simulation execution unit of each client so as to execute each simulation task in batch. The embodiment of the invention adopts a pipeline scheduling mode to generate the simulation tasks, controls the generation and the execution of the simulation tasks of a plurality of simulation scenes, is suitable for the operation requirements of a large number of simulation scenes, can efficiently execute the simulation tasks on a distributed system, and can ensure the affairs of the calculation result.
Description
Technical Field
The invention relates to the technical field of computer simulation, in particular to a method and a device for scheduling a simulation task.
Background
Simulation services require the execution of simulation parallel computations for many scenarios. After the web front end selects the set of simulation scenarios, the tasks of the simulation calculations are generated at the web back end. And after the tasks run in parallel, carrying out convergence statistics on the simulation results. The back-end computing task does not guarantee transactional support of the computing task. When a single task is executed, the large and uniform computing nodes complete all single scene simulation computation.
The affairs of various back-end computing tasks cannot be guaranteed, and the correctness of the whole task is reduced when part of computing fails. If the tasks of simulation calculation needing to be executed are more, the problems of low simulation calculation speed, low efficiency and the like exist. The method has frequent new requirements in a simulation service scene, and the existing scheme has high expansion support cost and insufficient flexibility.
Disclosure of Invention
The embodiment of the invention provides a method and a device for scheduling a simulation task, which are used for solving one or more technical problems in the prior art.
In a first aspect, an embodiment of the present invention provides a method for scheduling a simulation task, including:
receiving each simulation task initiation request aiming at a plurality of simulation scenes;
calling each simulation operator by adopting a pipeline scheduling mode to initiate a request aiming at each simulation task to generate each simulation task;
and sending each simulation task to a simulation execution unit of each client so as to execute each simulation task in batch.
In one embodiment, receiving simulation task initiation requests for a plurality of simulation scenarios includes:
receiving each simulation task initiation request sent by a front end FE through a hypertext transfer protocol HTTP interface; or
And receiving each simulation task initiation request sent by each client by using a Remote Procedure Call (RPC) interface.
In one embodiment, invoking each simulation operator in a pipeline scheduling manner to initiate a request for each simulation task to generate each simulation task includes:
calling a sequencing operator to set the batch execution sequence of each simulation task and generate an identifier of each simulation task;
calling a file transfer operator to obtain files required by executing each simulation task by using the identification of each simulation task;
and calling a task partitioning operator to fill the metadata information of each simulation task by using the file to obtain each executable simulation task.
In one embodiment, sending each simulation task to a simulation execution unit of each client to execute each simulation task in batch comprises:
and calling a task initiating operator to initiate each simulation task to the simulation execution unit of each client, and executing the received simulation task through the task engine of each simulation execution unit.
In one embodiment, the method further comprises:
calling a tracking operator to track the execution state and result of each simulation task, and dividing each simulation task according to the batch identification;
calling a report operator to converge the execution state of each simulation task according to the batch identification to obtain a report of the execution state of each simulation task;
and calling a statistical engine operator to perform statistics on the execution states of all simulation tasks of the whole system, and storing the statistics into a distributed database.
In one embodiment, the method further comprises:
and performing information interaction with garbage collection, a control panel, a distributed file system and a sentry tower of a Task Manager (TM) and each operator.
In a second aspect, an embodiment of the present invention provides a simulation task scheduling apparatus, including:
the simulation system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving each simulation task initiation request aiming at a plurality of simulation scenes;
the generating module is used for calling each simulation operator by adopting a pipeline scheduling mode to initiate a request aiming at each simulation task and generating each simulation task;
and the sending module is used for sending each simulation task to the simulation execution unit of each client so as to execute each simulation task in batch.
In one embodiment, the receiving module is further configured to:
receiving each simulation task initiation request sent by a front end FE through a hypertext transfer protocol HTTP interface; or
And receiving each simulation task initiation request sent by each client by using a Remote Procedure Call (RPC) interface.
In one embodiment, the generating module comprises:
the sequencing submodule is used for calling a sequencing operator to set the batch execution sequence of each simulation task and generate the identifier of each simulation task;
the file transfer submodule is used for calling a file transfer operator to obtain files required by executing each simulation task by utilizing the identification of each simulation task;
and the task division submodule is used for calling a task division operator to fill the metadata information of each simulation task by using the file to obtain each executable simulation task.
In one embodiment, the sending module comprises:
and the task initiating submodule is used for calling a task initiating operator to initiate each simulation task to the simulation execution unit of each client, and executing the received simulation task through the task engine of each simulation execution unit.
In one embodiment, the apparatus further comprises:
the tracking module is used for calling a tracking operator to track the execution state and the result of each simulation task and divide each simulation task according to the batch identification;
the report module is used for calling a report operator to converge the execution state of each simulation task according to the batch identification to obtain a report of the execution state of each simulation task;
and the statistical module is used for calling a statistical engine operator to perform statistics on the execution states of all simulation tasks of the whole system and storing the statistics into the distributed database.
In one embodiment, the apparatus further comprises:
and the task management module is used for performing information interaction with the garbage collection and control panel of the task manager TM, the distributed file system, the sentry tower and each operator.
In a third aspect, an embodiment of the present invention provides a simulation task scheduling device, where functions of the device may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions.
In one possible design, the apparatus structurally includes a processor and a memory, the memory is used for storing a program for supporting the apparatus to execute the above simulation task scheduling method, and the processor is configured to execute the program stored in the memory. The apparatus may also include a communication interface for communicating with other devices or a communication network.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium for storing computer software instructions for a simulation task scheduling apparatus, which includes a program for executing the simulation task scheduling method.
One of the above technical solutions has the following advantages or beneficial effects: the simulation tasks are generated in a pipeline scheduling mode, the generation and the execution of the multiple simulation tasks of multiple simulation scenes are controlled, the method is suitable for the operation requirements of large batches of simulation scenes, the simulation tasks can be executed efficiently on a distributed system, and the transactional nature of a calculation result can be guaranteed.
The foregoing summary is provided for the purpose of description only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present invention will be readily apparent by reference to the drawings and following detailed description.
Drawings
In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are therefore not to be considered limiting of its scope.
FIG. 1 shows a flow chart of a method of scheduling simulation tasks according to an embodiment of the invention.
FIG. 2 is a flow chart illustrating a method for scheduling simulation tasks according to an embodiment of the present invention.
Fig. 3 is a schematic diagram illustrating an application example of the simulation task scheduling method according to an embodiment of the present invention.
Fig. 4 is a schematic diagram illustrating another application example of the simulation task scheduling method according to the embodiment of the present invention.
Fig. 5 is a block diagram illustrating a structure of an emulation task scheduling apparatus according to an embodiment of the present invention.
Fig. 6 is a block diagram illustrating a structure of an emulation task scheduling apparatus according to an embodiment of the present invention.
Fig. 7 is a block diagram illustrating a structure of an emulation task scheduling apparatus according to an embodiment of the present invention.
Detailed Description
In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
FIG. 1 shows a flow chart of a method of scheduling simulation tasks according to an embodiment of the invention. As shown in fig. 1, the simulation task scheduling method may include a determination process, and specifically may include:
step S11, receiving each simulation task initiation request for a plurality of simulation scenarios.
And step S12, calling each simulation operator by adopting a pipeline scheduling mode to initiate a request aiming at each simulation task, and generating each simulation task.
Step S13, sending each simulation task to the simulation execution unit of each client, so as to execute each simulation task in batch.
In one example, the pipeline scheduling approach may be implemented using a pipeline scheduling framework of a Task Manager (TM). TM-based transactional semantics may support just-once (exact once) semantics. In a distributed system, exact once may be understood as a receiver (receiver) that receives only once for a message (message). The pipeline scheduling framework supports horizontal expansion, and hundreds of thousands of simulation scenes can be executed on tens of thousands of servers at the same time.
In the embodiment of the present invention, the server may receive an emulation task initiation request from a Front End (FE) or a Client (Client). Each simulation task initiation request may correspond to a simulation scenario. The server may receive multiple simulation task initiation requests for multiple simulation scenarios simultaneously. Basic information of the simulation task, such as metadata (meta) and the like, may be included in the simulation task initiation request. The server can complement the meta information to generate a simulation task that can be executed. And then sends the simulation tasks to relevant clients. The clients respectively execute the tasks, and the simulation tasks are executed in batch.
The simulation tasks are generated in a pipeline scheduling mode, the generation and the execution of the multiple simulation tasks of multiple simulation scenes are controlled, the method is suitable for the operation requirements of large batches of simulation scenes, the simulation tasks can be executed efficiently on a distributed system, and the transactional nature of a calculation result can be guaranteed.
In one embodiment, as shown in fig. 2, step S11 includes:
step S21, receiving each simulation task initiation request sent by a Web end (FE) through a hypertext transfer protocol (HTTP) interface; or
Step S22, receiving each simulation task initiation request sent by each client using a Remote Procedure Call (RPC) interface. The client can send a simulation task initiation request to the server in a command line mode.
In one embodiment, step S12 includes:
and step S23, calling a sequencing operator to set the batch execution sequence of each simulation task and generate the identifier of each simulation task.
And step S24, calling a file transfer operator to acquire files required by executing each simulation task by using the identification of each simulation task.
And step S25, calling a task dividing operator to fill the metadata information of each simulation task by using the file to obtain each executable simulation task.
In generating the simulation tasks, the server may call a sequencing operator to set a batch execution order of the plurality of simulation tasks and generate an identification of the simulation tasks. And then calling a file transfer operator, and searching files corresponding to the simulation tasks in the file system by using the identifications of the simulation tasks. After the search is completed, the information such as meta of the simulation tasks is correspondingly filled in the files, and a plurality of executable simulation tasks are generated.
In one embodiment, step S13 includes:
step S26, invoking a task initiating operator to initiate each simulation task to the simulation execution unit of each client, and executing the received simulation task by the task engine of each simulation execution unit.
The server can call the task initiating operator, determine the client end for executing the simulation task according to the identification of each simulation task, and correspondingly send the simulation task to the client end. The client side is provided with an emulation execution unit. The task engine of the simulation execution unit may execute the received simulation task. The server may send several simulation tasks in batches to multiple clients at the same time. After receiving the simulation tasks, the client can execute the simulation tasks according to the previously determined execution sequence. Multiple clients may execute tasks in parallel, thus enabling batch execution of simulation tasks.
In one embodiment, the method further comprises:
and step S27, calling a tracking operator to track the execution state and the result of each simulation task, and dividing each simulation task according to the batch identification.
And step S28, calling a report operator to converge the execution states of the simulation tasks according to the batch identification to obtain a report of the execution states of the simulation tasks.
And step S29, calling a statistic engine operator to count the execution states of all simulation tasks of the whole system, and storing the calculation results into a distributed database.
The server can call a tracking operator to track the execution state and the result of each simulation task and divide the simulation tasks according to the batch identification of each task. For example, tasks performed by the same batch have the same batch identification. The report operator can assemble the execution state of each task according to the batch identifier to obtain a report of the execution state of the batch task. In addition, the statistical engine operator can perform statistics on tasks of all batches in the whole system, or perform statistics on tasks of a plurality of specified batches to obtain a statistical result. These statistics are stored in a distributed database such as MySql, Palo, UDW, etc.
In one embodiment, the method further comprises: and performing information interaction with a garbage recoverer of the TM, a control panel, a distributed file system, a sentry tower and each operator.
And in the process of generating and executing the task, the TM is used for interacting with each operator. For example, when a file transfer operator is called to obtain a file needed to perform a task, the file can be obtained from a TM distributed file system, such as an AFS. The junk files generated in the process of generating and executing the tasks can be released through a junk retriever (Deleter). In addition, a TM control panel (Dashboard) can view detailed status of tasks within the TM, and perform detailed operations such as add-drop and modify check on tasks and Meta configurations. The TM sentinel towers can count and gather various state indexes of the assembly line in real time.
In an application example, referring to fig. 3, a process for executing simulation tasks in batch by using a pipeline framework may include the following steps:
1. several scenes can be selected at one time in a page of a web Front End (FE), and a batch of simulation task initiation requests are sent to a back end based on an HTTP protocol, wherein each request corresponds to one scene. The emulation task initiation request includes information such as lightweight metadata (meta) and the like.
2. The web backend, e.g., server, receives each simulation task initiation request. And calling a simulation operator to complete the parameter information and the configuration information of the requests, and generating a corresponding simulation task for each request.
3. And calling the simulation operator to send a simulation task to a simulation execution unit of the user to complete simulation calculation.
4. And calling a simulation operator to track the calculation results of each simulation task, and converging the calculation results.
5. In the whole process, a Task Master (TM) pipeline scheduling framework is adopted, and the semantics of actionly once is supported based on the transactional semantics of TM. Meanwhile, each stage of the assembly line can be disassembled flexibly (such as adding, deleting and expanding).
By adopting the simulation task scheduling method of the embodiment of the invention, a distributed stream type computing framework which meets just once (exact once) semantics can be established, and a simulation computing service logic is implemented based on the framework. The distributed streaming computing framework supports horizontal expansion, and can simultaneously execute hundreds of thousands of simulation scenes on tens of thousands of servers.
Referring to fig. 4, an example of a business computation process of a batch task is described in detail as follows:
first, a front end (dreamland FE) transmits a simulation task initiation request to a server (simmtrlservice) through an HTTP interface. Or sending an emulation task initiation request to the SimTrlService by using a command line at a Client (SimTrl Client).
And secondly, calling a corresponding simulation operator by the server to perform simulation calculation. The emulation operators may include, for example, an order (Arbiter) operator, a file transfer (FileTransfer) operator, a task partition (Splitter) operator, a task launch (Launcher) operator, a trace (Tracker) operator, a report (Reporter) operator, a statistics engine (StatEngine) operator, and the like.
Specifically, the Arbiter operator may set an order of executing the batch tasks, define an Identifier (ID) of each task, and ensure the transactional property of the overall computation. The FileTransfer operator obtains the files needed to execute the task. And filling information such as meta by using the Splitter operator by using the file to obtain executable tasks. Then, the Laucher operator simulates to the user an execution unit, for example: RosClient, Normandy, Matrix, etc. initiate the task. The tasks are executed by the Task Engine (Task Engine) of these units. The Tracker operator tracks the execution state and result of the task, and divides the task by lot ID (shading by regression _ ID). And then the Reporter operator gathers according to the batch ID to obtain a report. And after the StatEngine operator counts the task execution state of the whole system, storing the statistic into a distributed database such as MySql, Palo, UDW and the like.
Among them, meta, results, and the like stored in the database (MongoDB). In the execution process of the backend task, each module of the TM, such as a garbage collector (Deleter), a control panel (Dashboard), a distributed file system (AFS, for example), and a sentry tower, performs information interaction with each operator to assist each operator in completing the simulation process.
In fig. 4, the simmtrlservice has a half-work (worker) and half-service (service) feature. And each operator called by the SimTrlService belongs to the working module. dreamland FE, SimTrlService, user simulation execution unit, MongoDB, distributed database and the like belong to external modules. The TM belongs to a streaming basic service framework. Wherein, from SimClService to Arbiter, TM interfaces are adopted among operators and marked by solid arrows. From Splitter, Reporter to MongoDB, MongoDB to SimTrlService, StatEngine to the distributed database, using database interfaces, represented by dashed arrows. From Laucher to the execution unit, and from the execution unit to Tracker, Platform-as-a-Service (PaaS) interfaces are used, as indicated by the dotted arrows.
Fig. 5 is a block diagram illustrating a structure of an emulation task scheduling apparatus according to an embodiment of the present invention. As shown in fig. 5, the apparatus may include:
a receiving module 51, configured to receive each simulation task initiation request for a plurality of simulation scenarios;
the generating module 52 is configured to invoke each simulation operator to initiate a request for each simulation task by using a pipeline scheduling manner, so as to generate each simulation task;
and a sending module 53, configured to send each simulation task to the simulation execution unit of each client, so as to execute each simulation task in batch.
In one embodiment, the receiving module 51 is further configured to:
receiving each simulation task initiation request sent by a front end FE through a hypertext transfer protocol HTTP interface; or
And receiving each simulation task initiation request sent by each client by using a Remote Procedure Call (RPC) interface.
In one embodiment, as shown in fig. 6, the generating module 52 includes:
the sequencing submodule 61 is used for calling a sequencing operator to set the batch execution sequence of each simulation task and generate an identifier of each simulation task;
the file transfer submodule 62 is configured to invoke a file transfer operator to obtain files required for executing each simulation task by using the identifier of each simulation task;
and the task division submodule 63 is configured to invoke a task division operator to fill the metadata information of each simulation task with the file, so as to obtain each executable simulation task.
In one embodiment, the sending module 53 includes:
the task initiating submodule 64 is configured to invoke a task initiating operator to initiate each simulation task to the simulation execution unit of each client, and execute the received simulation task through the task engine of each simulation execution unit.
In one embodiment, the apparatus further comprises:
the tracking module 65 is configured to invoke a tracking operator to track an execution state and a result of each simulation task, and divide each simulation task according to the batch identifier;
the report module 66 is configured to invoke a report operator to aggregate the execution states of the simulation tasks according to the batch identifiers, so as to obtain a report of the execution states of the simulation tasks;
and the statistical module 67 is used for calling a statistical engine operator to perform statistics on the execution states of all simulation tasks of the whole system and storing the statistics into the distributed database.
In one embodiment, the apparatus further comprises:
and the task management module 68 is used for performing information interaction with the garbage collection, the control panel, the distributed file system and the sentry tower of the task manager TM and each operator.
The functions of each module in each apparatus in the embodiments of the present invention may refer to the corresponding description in the above method, and are not described herein again.
Fig. 7 is a block diagram illustrating a structure of an emulation task scheduling apparatus according to an embodiment of the present invention. As shown in fig. 7, the apparatus includes: a memory 910 and a processor 920, the memory 910 having stored therein computer programs operable on the processor 920. The processor 920 implements the transaction commit method in the above embodiments when executing the computer program. The number of the memory 910 and the processor 920 may be one or more.
The device also includes:
and a communication interface 930 for communicating with an external device to perform data interactive transmission.
If the memory 910, the processor 920 and the communication interface 930 are implemented independently, the memory 910, the processor 920 and the communication interface 930 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 7, but this is not intended to represent only one bus or type of bus.
Optionally, in an implementation, if the memory 910, the processor 920 and the communication interface 930 are integrated on a chip, the memory 910, the processor 920 and the communication interface 930 may complete communication with each other through an internal interface.
An embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, and the computer program is used for implementing the method of any one of the above embodiments when being executed by a processor.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present invention, and these should be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (14)
1. A method for scheduling simulation tasks is characterized by comprising the following steps:
receiving each simulation task initiation request aiming at a plurality of simulation scenes;
calling each simulation operator by adopting a pipeline scheduling mode to initiate a request aiming at each simulation task to generate each simulation task;
and sending each simulation task to a simulation execution unit of each client so as to execute each simulation task in batch.
2. The method of claim 1, wherein receiving each simulation task initiation request for a plurality of simulation scenarios comprises:
receiving each simulation task initiation request sent by a front end FE through a hypertext transfer protocol HTTP interface; or
And receiving each simulation task initiation request sent by each client by using a Remote Procedure Call (RPC) interface.
3. The method of claim 1, wherein invoking each simulation operator in a pipeline scheduling manner initiates a request for each simulation task to generate each simulation task, comprising:
calling a sequencing operator to set the batch execution sequence of each simulation task and generate an identifier of each simulation task;
calling a file transfer operator to obtain files required by executing each simulation task by using the identification of each simulation task;
and calling a task partitioning operator to fill the metadata information of each simulation task by using the file to obtain each executable simulation task.
4. The method of claim 1, wherein sending each of the simulation tasks to a simulation execution unit of each of the clients for batch execution of each of the simulation tasks comprises:
and calling a task initiating operator to initiate each simulation task to the simulation execution unit of each client, and executing the received simulation task through the task engine of each simulation execution unit.
5. The method of any of claims 1 to 4, further comprising:
calling a tracking operator to track the execution state and result of each simulation task, and dividing each simulation task according to the batch identification;
calling a report operator to converge the execution state of each simulation task according to the batch identification to obtain a report of the execution state of each simulation task;
and calling a statistical engine operator to perform statistics on the execution states of all simulation tasks of the whole system, and storing the statistics into a distributed database.
6. The method of any of claims 1 to 4, further comprising:
and performing information interaction with garbage collection, a control panel, a distributed file system and a sentry tower of a Task Manager (TM) and each operator.
7. An apparatus for scheduling a simulation task, comprising:
the simulation system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving each simulation task initiation request aiming at a plurality of simulation scenes;
the generating module is used for calling each simulation operator by adopting a pipeline scheduling mode to initiate a request aiming at each simulation task and generating each simulation task;
and the sending module is used for sending each simulation task to the simulation execution unit of each client so as to execute each simulation task in batch.
8. The apparatus of claim 7, wherein the receiving module is further configured to:
receiving each simulation task initiation request sent by a front end FE through a hypertext transfer protocol HTTP interface; or
And receiving each simulation task initiation request sent by each client by using a Remote Procedure Call (RPC) interface.
9. The apparatus of claim 7, wherein the generating module comprises:
the sequencing submodule is used for calling a sequencing operator to set the batch execution sequence of each simulation task and generate the identifier of each simulation task;
the file transfer submodule is used for calling a file transfer operator to obtain files required by executing each simulation task by utilizing the identification of each simulation task;
and the task division submodule is used for calling a task division operator to fill the metadata information of each simulation task by using the file to obtain each executable simulation task.
10. The apparatus of claim 7, wherein the sending module comprises:
and the task initiating submodule is used for calling a task initiating operator to initiate each simulation task to the simulation execution unit of each client, and executing the received simulation task through the task engine of each simulation execution unit.
11. The apparatus of any one of claims 7 to 10, further comprising:
the tracking module is used for calling a tracking operator to track the execution state and the result of each simulation task and divide each simulation task according to the batch identification;
the report module is used for calling a report operator to converge the execution state of each simulation task according to the batch identification to obtain a report of the execution state of each simulation task;
and the statistical module is used for calling a statistical engine operator to perform statistics on the execution states of all simulation tasks of the whole system and storing the statistics into the distributed database.
12. The apparatus of any one of claims 7 to 10, further comprising:
and the task management module is used for performing information interaction with the garbage collection and control panel of the task manager TM, the distributed file system, the sentry tower and each operator.
13. An apparatus for scheduling a simulation task, comprising:
one or more processors;
storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-6.
14. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811277421.3A CN111199079B (en) | 2018-10-30 | 2018-10-30 | Simulation task scheduling method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811277421.3A CN111199079B (en) | 2018-10-30 | 2018-10-30 | Simulation task scheduling method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111199079A true CN111199079A (en) | 2020-05-26 |
| CN111199079B CN111199079B (en) | 2023-12-19 |
Family
ID=70745885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811277421.3A Active CN111199079B (en) | 2018-10-30 | 2018-10-30 | Simulation task scheduling method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111199079B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112639734A (en) * | 2020-12-10 | 2021-04-09 | 华为技术有限公司 | Simulation task scheduling method, execution method, simulation implementation method and device |
| CN113254157A (en) * | 2021-06-04 | 2021-08-13 | 北京世冠金洋科技发展有限公司 | Batch simulation control method and device and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070129929A1 (en) * | 2005-12-02 | 2007-06-07 | Hiroaki Nakamura | System simulation using multi-tasking computer code |
| CN201298233Y (en) * | 2008-10-22 | 2009-08-26 | 西北电网有限公司 | An electrical power system electro-magnetism transient distributed simulation device |
| CN102393710A (en) * | 2011-10-28 | 2012-03-28 | 无锡市星亿涂装环保设备有限公司 | Multitask production simulation schedule control system for production line travelling crane and method thereof |
| US20130144907A1 (en) * | 2011-12-06 | 2013-06-06 | Microsoft Corporation | Metadata extraction pipeline |
| CN105843189A (en) * | 2016-04-09 | 2016-08-10 | 北京化工大学 | Simplified simulation model based high efficient scheduling rule choosing method for use in semiconductor production lines |
| US20170305448A1 (en) * | 2014-11-20 | 2017-10-26 | Hitachi, Ltd. | Degradation estimation system of railroad ground equipment and method thereof |
| US20180189431A1 (en) * | 2017-12-18 | 2018-07-05 | Hosni I. Adra | Dynamic simulation models constantly adapting to the changes of complex systems |
-
2018
- 2018-10-30 CN CN201811277421.3A patent/CN111199079B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070129929A1 (en) * | 2005-12-02 | 2007-06-07 | Hiroaki Nakamura | System simulation using multi-tasking computer code |
| CN201298233Y (en) * | 2008-10-22 | 2009-08-26 | 西北电网有限公司 | An electrical power system electro-magnetism transient distributed simulation device |
| CN102393710A (en) * | 2011-10-28 | 2012-03-28 | 无锡市星亿涂装环保设备有限公司 | Multitask production simulation schedule control system for production line travelling crane and method thereof |
| US20130144907A1 (en) * | 2011-12-06 | 2013-06-06 | Microsoft Corporation | Metadata extraction pipeline |
| US20170305448A1 (en) * | 2014-11-20 | 2017-10-26 | Hitachi, Ltd. | Degradation estimation system of railroad ground equipment and method thereof |
| CN105843189A (en) * | 2016-04-09 | 2016-08-10 | 北京化工大学 | Simplified simulation model based high efficient scheduling rule choosing method for use in semiconductor production lines |
| US20180189431A1 (en) * | 2017-12-18 | 2018-07-05 | Hosni I. Adra | Dynamic simulation models constantly adapting to the changes of complex systems |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112639734A (en) * | 2020-12-10 | 2021-04-09 | 华为技术有限公司 | Simulation task scheduling method, execution method, simulation implementation method and device |
| CN113254157A (en) * | 2021-06-04 | 2021-08-13 | 北京世冠金洋科技发展有限公司 | Batch simulation control method and device and electronic equipment |
| CN113254157B (en) * | 2021-06-04 | 2023-07-28 | 北京世冠金洋科技发展有限公司 | Batch simulation control method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111199079B (en) | 2023-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110290189B (en) | Container cluster management method, device and system | |
| CN109561171B (en) | Configuration method and device of virtual private cloud service | |
| CN109862065B (en) | File downloading method and device and electronic equipment | |
| CN108170612B (en) | Automatic testing method and device and server | |
| CN108089913B (en) | Virtual machine deployment method of super-fusion system | |
| CN109886693B (en) | Consensus realization method, device, equipment and medium for block chain system | |
| CN111262726B (en) | Configuration information updating method and device and computer readable storage medium | |
| CN110765024A (en) | Simulation test method, simulation test device, electronic equipment and computer-readable storage medium | |
| CN110830234B (en) | User traffic distribution method and device | |
| CN110310139B (en) | Data delivery method and data delivery engine device | |
| CN109117252A (en) | Method, system and the container cluster management system of task processing based on container | |
| CN109104368B (en) | Connection request method, device, server and computer readable storage medium | |
| CN111199079A (en) | Simulation task scheduling method and device | |
| CN110704376A (en) | Log file saving method and device | |
| CN112468545A (en) | Cloud host creation method, device, system, equipment and readable storage medium | |
| CN109634752A (en) | A kind of client request processing method and system based on page gateway | |
| WO2021013185A1 (en) | Virtual machine migration processing and strategy generation method, apparatus and device, and storage medium | |
| CN113268308B (en) | Information processing method, device and storage medium | |
| CN112612496A (en) | Intelligent contract operation management method, device, equipment and computer storage medium | |
| CN108334410A (en) | A kind of distributed application program client light weight method and computer equipment | |
| CN116974874A (en) | Database testing method and device, electronic equipment and readable storage medium | |
| CN110493891B (en) | Resource allocation method, device and system | |
| CN115984022A (en) | Unified account checking method and device for distributed payment system | |
| CN103425707A (en) | Data analyzing method and data analyzing device | |
| CN109873731B (en) | Test method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211014 Address after: 105 / F, building 1, No. 10, Shangdi 10th Street, Haidian District, Beijing 100085 Applicant after: Apollo Intelligent Technology (Beijing) Co.,Ltd. Address before: 100085 Baidu Building, 10 Shangdi Tenth Street, Haidian District, Beijing Applicant before: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |