CN111797110A - Method and device for generating scheduling model, computer equipment and storage medium - Google Patents
Method and device for generating scheduling model, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111797110A CN111797110A CN202010579183.2A CN202010579183A CN111797110A CN 111797110 A CN111797110 A CN 111797110A CN 202010579183 A CN202010579183 A CN 202010579183A CN 111797110 A CN111797110 A CN 111797110A
- Authority
- CN
- China
- Prior art keywords
- scheduling unit
- scheduling
- database
- priority
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 13
- 238000012545 processing Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for generating a scheduling model, and relates to the field of computers. The method comprises the following steps: determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database; selecting a target scheduling unit from at least one scheduling unit according to the priority; acquiring characteristic data of a target scheduling unit, and generating a scheduling model according to the characteristic data to complete a task to be executed; wherein the characteristic data at least comprises identification information and state information. The invention can control the scheduling of the data in the database with finer granularity, thereby improving the accuracy and efficiency of the data scheduling in the database.
Description
Technical Field
The present invention relates to the field of computers, and in particular, to a database scheduling method, apparatus, computer device, and storage medium.
Background
At present, big data brings massive, various and unstructured data, and the data can be accurately displayed and are realized based on a one-by-one streaming task for processing the data behind. However, the conventional task scheduling method can only design a streaming task for a specific problem, has low universality, is difficult to control all data in the database in a fine-grained manner, and particularly when facing a large number of complex tasks, increases the cost due to low multiplexing rate, and greatly reduces the data processing efficiency and stability.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for generating a scheduling model, which can improve the accuracy and efficiency of data scheduling in a database, and are easy to manage and high in universality.
To achieve the above object, according to an aspect of an embodiment of the present invention, a method of generating a scheduling model is provided.
The method for generating the scheduling model comprises the following steps:
determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database;
selecting a target scheduling unit from at least one scheduling unit according to the priority;
acquiring characteristic data of a target scheduling unit, and generating a scheduling model according to the characteristic data to complete a task to be executed; wherein the characteristic data at least comprises identification information and state information.
Optionally, determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database includes:
determining the initial priority of each scheduling unit in the database according to the task to be executed;
receiving and analyzing the temporary scheduling instruction to adjust the priority of the scheduling unit corresponding to the temporary scheduling instruction;
and updating the priority of each scheduling unit in the database according to the initial priority and the priority of the scheduling unit corresponding to the adjusted temporary scheduling instruction.
Optionally, the step of obtaining the feature data of the target scheduling unit includes:
determining identification information of a target scheduling unit;
positioning the storage position of the target scheduling unit in the database according to the identification information;
and acquiring the state information of the target scheduling unit according to the storage position.
Optionally, before obtaining the feature data of the target scheduling unit, the method further includes:
configuring an initial identifier for each scheduling unit in at least one scheduling unit;
encrypting the initial identifier of each scheduling unit to obtain an encrypted identifier;
and determining the storage position and the identification information of each scheduling unit according to the encrypted identification.
Optionally, after obtaining the feature data of the target scheduling unit and generating the scheduling model according to the feature data to complete the task to be executed, the method further includes:
acquiring a scheduling result and updating state information of a target scheduling unit, wherein the scheduling result at least comprises data of the target scheduling unit after a task to be executed is completed and attribute data of the task to be executed;
storing the scheduling result and the updated state information into a database;
and updating the priority of at least one scheduling unit according to the scheduling result.
Optionally, before determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database, the method further includes:
determining the number of at least one scheduling unit and the number of tables stored in a database;
determining a storage strategy of at least one scheduling unit according to the number of the at least one scheduling unit and the number of the storage tables in the database;
and storing the at least one scheduling unit into a storage table in the database according to the storage strategy.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided an apparatus for generating a scheduling model.
The device for generating the scheduling model of the embodiment of the invention comprises:
setting a module: the system comprises a database, a scheduling unit and a control unit, wherein the database is used for storing at least one scheduling unit;
a selecting module: the scheduling unit is used for selecting a target scheduling unit from at least one scheduling unit according to the priority;
a generation module: the system comprises a target scheduling unit, a scheduling model and a task execution module, wherein the target scheduling unit is used for acquiring characteristic data of the target scheduling unit and generating the scheduling model according to the characteristic data so as to complete a task to be executed; wherein the characteristic data at least comprises identification information and state information.
Optionally, the setting module is further configured to determine an initial priority of each scheduling unit in the database according to the task to be executed; receiving and analyzing the temporary scheduling instruction to adjust the priority of the scheduling unit corresponding to the temporary scheduling instruction; and updating the priority of each scheduling unit in the database according to the initial priority and the priority of the scheduling unit corresponding to the adjusted temporary scheduling instruction.
Optionally, the generating module is further configured to: determining identification information of a target scheduling unit; positioning the storage position of the target scheduling unit in the database according to the identification information; and acquiring the state information of the target scheduling unit according to the storage position.
Optionally, generating the scheduling model further comprises: the configuration module is used for configuring an initial identifier for each scheduling unit in at least one scheduling unit, carrying out encryption processing to obtain an encrypted identifier, and determining the storage position and the identifier information of each scheduling unit according to the encrypted identifier.
Optionally, generating the scheduling model further comprises:
an update module: the scheduling unit is used for acquiring a scheduling result, updating the state information of the target scheduling unit, storing the scheduling result and the updated state information into a database, and updating the priority of at least one scheduling unit according to the scheduling result;
the scheduling result at least comprises data of the target scheduling unit after the task to be executed is completed and attribute data of the task to be executed.
Optionally, generating the scheduling model further comprises:
a storage module: the method comprises the steps of determining the number of at least one scheduling unit and the number of tables stored in a database; determining a storage strategy of at least one scheduling unit according to the number of the at least one scheduling unit and the number of the storage tables in the database; and storing the at least one scheduling unit into a storage table in the database according to the storage strategy.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided an electronic apparatus.
The electronic device of the embodiment of the invention comprises: one or more processors; a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method of generating a scheduling model of any of the above.
To achieve the above object, according to a further aspect of the embodiments of the present invention, there is provided a computer readable medium having a computer program stored thereon, wherein the program is configured to implement any one of the above methods for generating a scheduling model when executed by a processor.
One embodiment of the above invention has the following advantages or benefits:
(1) the scheduling unit is used as a basic unit of data scheduling, and the scheduling of data in the database can be controlled in the finest granularity, so that the accuracy and the efficiency of data scheduling in the database are improved, and the management is easy;
(2) the invention can realize the scheduling work required by different tasks by adjusting the priority of each scheduling unit according to the requirements of the tasks to be processed, is widely suitable for various application scenes with task scheduling requirements, and has higher universality.
(3) Each scheduling unit has own identification information, so that the state of each scheduling unit can be easily inquired in real time; and the scheduling results including state information, attribute data and the like of each scheduled scheduling unit are stored in the database, so that the scheduling history of the database can be conveniently checked, the life cycle of a scheduling task can be traced, and when a problem is found in the scheduling process, the problem can be easily checked according to the scheduling history.
(4) According to the invention, each scheduling unit is stored in each data table of the database according to a certain storage strategy, so that the scheduling speed and accuracy are improved.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a main flow of a method of generating a scheduling model according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a main flow of a method for updating the priority of each scheduling unit by a scheduling model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the main modules of an apparatus for generating a scheduling model according to an embodiment of the present invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 5 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of a main flow of a method for generating a scheduling model according to an embodiment of the present invention, and as shown in fig. 1, the method for generating a scheduling model according to an embodiment of the present invention mainly includes:
step S101: determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database;
step S102: selecting a target scheduling unit from at least one scheduling unit according to the priority;
step S103: acquiring characteristic data of a target scheduling unit, and generating a scheduling model according to the characteristic data to complete a task to be executed; wherein the characteristic data at least comprises identification information and state information.
According to the embodiment of the invention, the scheduling unit is used as a basic unit of data scheduling, and the scheduling of data in the database can be controlled in the finest granularity, so that the accuracy and the efficiency of data scheduling in the database are improved, and the management is easy; and the scheduling work required by different tasks can be realized by adjusting the priority of each scheduling unit according to the requirements of the tasks to be processed, so that the method is widely suitable for various application scenes with task scheduling requirements, and has high universality.
In step S101, the present embodiment is applied to a database, and a scheduling unit is used as a basic unit of data scheduling, so as to control scheduling of data in the database with the finest granularity. In addition, due to the fact that requirements of the tasks to be executed are different, the scheduling units needing scheduling are different, the priority of each scheduling unit is determined according to the tasks to be executed, the scheduling requirements of the tasks can be met by adjusting the priority according to the different tasks to be executed, and a new model which is the same as a traditional scheduling model and can be designed only according to a specific problem is not needed.
In the process of determining the priority of each scheduling unit according to the tasks to be executed based on at least one scheduling unit stored in the database, the priority sequence of at least one scheduling unit is used as a necessary condition for selecting a target scheduling unit. In general, a top N with the highest current priority is screened out by a model and then aggregated together to serve as a task selected once, in the embodiment of the invention, the size of N is inversely proportional to the average completion time of the task, and the shorter the time, the larger N is. Specifically, the priority of the scheduling unit is different depending on the actual application scenario, for example, a batch of companies needs to be updated incrementally, but since the updating capability is limited and the updating cannot be traversed to all companies, the policy can be customized, and the priority of a batch of companies with a high probability of being changed by a worker is increased, so that more changed data is covered by using the limited updating capability. This priority setting strategy is flexible and can be set according to the actual application scenario.
In step S102, a target scheduling unit is selected from the at least one scheduling unit according to the priority of each scheduling unit determined in step S101. The target scheduling unit is generally a scheduling unit of topN with the highest priority required by the task.
In an embodiment of the present invention, selecting a target scheduling unit from at least one scheduling unit according to the priority may specifically include:
step S201: determining the initial priority of each scheduling unit in the database according to the task to be executed;
step S202: receiving and analyzing the temporary scheduling instruction to adjust the priority of the scheduling unit corresponding to the temporary scheduling instruction;
step S203: and updating the priority of each scheduling unit in the database according to the initial priority and the priority of the scheduling unit corresponding to the adjusted temporary scheduling instruction.
In the embodiment of the invention, the scheduling model only determines an initial priority, before the target scheduling unit is selected, if the temporary scheduling instruction appears, the temporary scheduling instruction can be received and analyzed in time, namely the priority of each scheduling unit in the final database is updated and formed according to the priority of the scheduling unit corresponding to the adjusted temporary scheduling instruction, and the priority of the corresponding scheduling unit is adjusted in time according to the temporary scheduling instruction, so that the flexibility of the scheduling model is embodied. Wherein the temporary scheduling instruction only adjusts the priority of the new target scheduling unit, and determining the final priority (i.e. the updated priority) further needs to be combined with the previously determined initial priority. The temporary scheduling instructions generally include: and the priority of the scheduling unit needing to be temporarily scheduled is adjusted to be the highest, and/or the priority of the scheduling unit needing not to be temporarily scheduled is adjusted to be the lowest.
Preferably, in step S103, the characteristic data of the target scheduling unit includes, but is not limited to, identification information and status information, where the identification information is data uniquely identifying the target scheduling unit and may be a location identifier or an identification code of the scheduling unit. The status information indicates that the target scheduling unit is in a scheduling, idle, or other status. And generating a scheduling model according to the characteristic data to perform scheduling operation, thereby completing the task to be executed.
The obtaining of the feature data of the target scheduling unit may specifically include: step S301: determining identification information of a target scheduling unit; step S302: positioning the storage position of the target scheduling unit in the database according to the identification information; step S303: and acquiring the state information of the target scheduling unit according to the storage position.
In an embodiment of the present invention, before obtaining the feature data of the target scheduling unit and generating the scheduling model according to the feature data, the method may further include: step S111: configuring an initial identifier for each scheduling unit in at least one scheduling unit; step S112: encrypting the initial identifier of each scheduling unit to obtain an encrypted identifier; step S113: and determining the storage position and the identification information of each scheduling unit according to the encrypted identification.
In an embodiment of the present invention, the storage location and the identification information of each scheduling unit in at least one scheduling unit are unique, and assuming that the unique identification of each scheduling unit is a string of numbers, the string of numbers may be encrypted by md5, and then two bits are taken as the location index of the scheduling unit stored in the database, for example, after "11233" is made into md5, the value is "60 e4ac6a656ef4b4fd82aaaf25f 14736", and the two bits are taken as "36", then it may be determined that the scheduling unit is stored in the No. 3 library No. 6 table. The storage position and the identification information of each scheduling unit are unique, so that the state query of each scheduling unit is facilitated, and the associated scheduling history is convenient to trace.
Fig. 2 is a schematic diagram of a main flow of a method for updating the priority of each scheduling unit by a scheduling model according to an embodiment of the present invention, and as shown in fig. 2, after obtaining feature data of a target scheduling unit and generating the scheduling model according to the feature data, the method may further update the priority of each scheduling unit in the scheduling model, and specifically includes:
step S104: acquiring a scheduling result and updating state information of a target scheduling unit, wherein the scheduling result at least comprises data of the target scheduling unit after a task to be executed is completed and attribute data of the task to be executed;
step S105: storing the scheduling result and the updated state information into a database;
step S106: and updating the priority of at least one scheduling unit according to the scheduling result.
In the embodiment of the present invention, the status information includes: in the scheduling operation, the scheduling operation is not in the process of scheduling; the attribute data includes task identification, execution time, associated task information, and the like. Therefore, whether the scheduling unit is in the scheduling state can be determined according to the state information, and when the scheduling unit is in the scheduling state, the scheduling unit can be scheduled again after the scheduling unit completes scheduling, namely the state information of the scheduling unit is updated to be not in the scheduling operation. In the embodiment of the invention, the related data of the task execution can be determined through the attribute data, so that developers can know the task execution state conveniently.
In step S105, the scheduling result including the status information, the attribute data, and the like of each scheduled scheduling unit is stored in the database, so that the scheduling history of the database can be conveniently checked to trace back the life cycle of a scheduling task, and when a problem is found in the scheduling process, the problem can be easily checked according to the scheduling history.
In step S106, the scheduling units vary according to the task to be processed, and the scheduling result after the previous scheduling changes the relevant data of the target scheduling unit, so that the priority of each scheduling unit corresponding to the next task to be processed varies.
In an embodiment of the present invention, before determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database, the method may further include: step S1001: determining the number of at least one scheduling unit and the number of data tables in a database; step S1002: determining a storage strategy of at least one scheduling unit according to the number of the at least one scheduling unit and the number of data tables in a database; step S1003: and storing at least one scheduling unit into a data table in a database according to the storage strategy.
In this embodiment, each scheduling unit is stored in each data table of the database according to a certain storage policy, so as to improve the speed and accuracy of scheduling. Preferably, the storage strategy is that each scheduling unit is uniformly stored in the database, so that the scheduling units can be selected for scheduling according to the speed of the actual processing task, and further, the equal probability selection of all the scheduling units is ensured.
Fig. 3 is a schematic diagram of main modules of an apparatus for generating a scheduling model according to an embodiment of the present invention, and as shown in fig. 3, an apparatus 300 for generating a scheduling model according to an embodiment of the present invention includes a setting module 301, a selecting module 302, and a generating module 303.
A setting module 301, configured to determine, based on at least one scheduling unit stored in the database, a priority of each scheduling unit according to a task to be executed;
a selecting module 302, configured to select a target scheduling unit from the at least one scheduling unit according to the priority;
the generating module 303 is configured to obtain feature data of the target scheduling unit, and generate a scheduling model according to the feature data to complete a task to be executed; wherein the characteristic data at least comprises identification information and state information.
According to the scheduling model of the embodiment of the invention, the scheduling operation is carried out, and the scheduling of the data in the database can be controlled in the finest granularity, so that the accuracy and the efficiency of the data scheduling in the database are improved, and the management is easy; and the method can realize the scheduling work required by different tasks, is widely suitable for various application scenes with task scheduling requirements, and has higher universality.
In an embodiment of the present invention, the setting module is further configured to determine an initial priority of each scheduling unit in the database according to the task to be executed; receiving and analyzing the temporary scheduling instruction to adjust the priority of the scheduling unit corresponding to the temporary scheduling instruction; and updating the priority of each scheduling unit in the database according to the initial priority and the priority of the scheduling unit corresponding to the adjusted temporary scheduling instruction.
In an embodiment of the present invention, the generating module is further configured to: determining identification information of a target scheduling unit; positioning the storage position of the target scheduling unit in the database according to the identification information; and acquiring the state information of the target scheduling unit according to the storage position.
In an embodiment of the present invention, the apparatus for generating a scheduling model further includes a configuration module, configured to configure an initial identifier for each scheduling unit in at least one scheduling unit, perform encryption processing to obtain an encrypted identifier, and determine a storage location and identifier information of each scheduling unit according to the encrypted identifier.
In an embodiment of the present invention, the apparatus for generating a scheduling model further includes an updating module, configured to obtain a scheduling result, update the state information of the target scheduling unit, store the scheduling result and the updated state information in a database, and update the priority of at least one scheduling unit according to the scheduling result. The scheduling result at least comprises data of the target scheduling unit after the task to be executed is completed and attribute data of the task to be executed.
In an embodiment of the present invention, the apparatus for generating a scheduling model further includes a storage module, configured to determine the number of at least one scheduling unit and the number of storage tables in a database; determining a storage strategy of at least one scheduling unit according to the number of the at least one scheduling unit and the number of the storage tables in the database; and storing the at least one scheduling unit into a storage table in the database according to the storage strategy.
Fig. 4 illustrates an exemplary system architecture 400 to which the method of generating a scheduling model or the apparatus for generating a scheduling model of an embodiment of the present invention may be applied.
As shown in fig. 4, the system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405. The network 404 serves as a medium for providing communication links between the terminal devices 401, 402, 403 and the server 405. Network 404 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 401, 402, 403 to interact with a server 405 over a network 404 to receive or send messages or the like. The terminal devices 401, 402, 403 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 405 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 401, 402, 403. The background management server can analyze and process the received data such as the product information inquiry request and feed back the processing result to the terminal equipment.
It should be noted that the method for generating the scheduling model provided by the embodiment of the present invention is generally executed by the server 405, and accordingly, the apparatus for generating the scheduling model is generally disposed in the server 405.
It should be understood that the number of terminal devices, networks, and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 5, shown is a block diagram of a computer system 500 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 501.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes an acquisition setting module, a selection module, and a generation module. The names of these modules do not constitute a limitation to the modules themselves in some cases, and for example, the setting module may also be described as a "module that determines the priority of each scheduling unit according to the task to be executed".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database; selecting a target scheduling unit from at least one scheduling unit according to the priority; acquiring characteristic data of a target scheduling unit, and generating a scheduling model according to the characteristic data to complete a task to be executed; wherein the characteristic data at least comprises identification information and state information.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of generating a scheduling model, comprising:
determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database;
selecting a target scheduling unit from the at least one scheduling unit according to the priority;
acquiring characteristic data of the target scheduling unit, and generating a scheduling model according to the characteristic data to complete the task to be executed; wherein the characteristic data at least comprises identification information and state information.
2. The method of claim 1, wherein the determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database comprises:
determining the initial priority of each scheduling unit in the database according to the task to be executed;
receiving and analyzing a temporary scheduling instruction to adjust the priority of a scheduling unit corresponding to the temporary scheduling instruction;
and updating the priority of each scheduling unit in the database according to the initial priority and the adjusted priority of the scheduling unit corresponding to the temporary scheduling instruction.
3. The method of claim 1, wherein the step of obtaining the characteristic data of the target scheduling unit comprises:
determining identification information of the target scheduling unit;
positioning the storage position of the target scheduling unit in the database according to the identification information;
and acquiring the state information of the target scheduling unit according to the storage position.
4. The method of claim 3, further comprising, prior to said obtaining the characteristic data of the target scheduling unit:
configuring an initial identifier for each scheduling unit in the at least one scheduling unit;
encrypting the initial identifier of each scheduling unit to obtain an encrypted identifier;
and determining the storage position and the identification information of each scheduling unit according to the encrypted identification.
5. The method according to claim 1, wherein after the obtaining the feature data of the target scheduling unit and generating a scheduling model according to the feature data to complete the task to be executed, the method further comprises:
acquiring a scheduling result and updating the state information of the target scheduling unit, wherein the scheduling result at least comprises data of the target scheduling unit after the task to be executed is completed and attribute data of the task to be executed;
storing the scheduling result and the updated state information into the database;
and updating the priority of the at least one scheduling unit according to the scheduling result.
6. The method according to any one of claims 1-5, wherein before determining the priority of each scheduling unit according to the task to be executed based on at least one scheduling unit stored in the database, further comprising:
determining the number of the at least one scheduling unit and the number of data tables in the database;
determining a storage strategy of the at least one scheduling unit according to the number of the at least one scheduling unit and the number of data tables in the database;
and storing the at least one scheduling unit into a data table in the database according to the storage strategy.
7. An apparatus that generates a scheduling model, comprising:
setting a module: the system comprises a database, a scheduling unit and a control unit, wherein the database is used for storing at least one scheduling unit;
a selecting module: the scheduling unit is used for selecting a target scheduling unit from the at least one scheduling unit according to the priority;
a generation module: the system comprises a target scheduling unit, a scheduling module and a task execution module, wherein the target scheduling unit is used for acquiring characteristic data of the target scheduling unit and generating a scheduling model according to the characteristic data so as to complete the task to be executed; wherein the characteristic data at least comprises identification information and state information.
8. The apparatus of claim 7, further comprising:
a storage module: for determining the number of the at least one scheduling unit and the number of data tables in the database; determining a storage strategy of the at least one scheduling unit according to the number of the at least one scheduling unit and the number of data tables in the database; and storing the at least one scheduling unit into a data table in the database according to the storage strategy.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010579183.2A CN111797110B (en) | 2020-06-23 | 2020-06-23 | Method, device, computer equipment and storage medium for generating scheduling model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010579183.2A CN111797110B (en) | 2020-06-23 | 2020-06-23 | Method, device, computer equipment and storage medium for generating scheduling model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111797110A true CN111797110A (en) | 2020-10-20 |
| CN111797110B CN111797110B (en) | 2024-01-26 |
Family
ID=72803087
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010579183.2A Active CN111797110B (en) | 2020-06-23 | 2020-06-23 | Method, device, computer equipment and storage medium for generating scheduling model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111797110B (en) |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030225820A1 (en) * | 2002-05-31 | 2003-12-04 | Microsoft Corporation | System and method for collecting and storing event data from distributed transactional applications |
| US20120222035A1 (en) * | 2011-02-25 | 2012-08-30 | Qualcomm Incorporated | Priority Inheritance in Multithreaded Systems |
| US20130152098A1 (en) * | 2011-12-12 | 2013-06-13 | International Business Machines Corporation | Task priority boost management |
| CN105512027A (en) * | 2015-11-26 | 2016-04-20 | 珠海多玩信息技术有限公司 | Process state monitor method and device |
| US20170142027A1 (en) * | 2015-11-13 | 2017-05-18 | Raytheon Company | Dynamic priority calculator for priority based scheduling |
| CN107016480A (en) * | 2016-01-28 | 2017-08-04 | 五八同城信息技术有限公司 | Method for scheduling task, apparatus and system |
| CN107102894A (en) * | 2017-04-07 | 2017-08-29 | 百度在线网络技术（北京）有限公司 | Method for scheduling task, device and system |
| CN107391198A (en) * | 2017-07-25 | 2017-11-24 | 广东欧珀移动通信有限公司 | Method for scheduling task and device, computer-readable recording medium, mobile terminal |
| CN107977268A (en) * | 2017-10-13 | 2018-05-01 | 北京百度网讯科技有限公司 | Method for scheduling task, device and the computer-readable recording medium of the isomerization hardware of artificial intelligence |
| CN108132837A (en) * | 2018-01-02 | 2018-06-08 | 中国工商银行股份有限公司 | A kind of distributed type assemblies dispatch system and method |
| CN108717378A (en) * | 2018-04-20 | 2018-10-30 | 中国气象局气象探测中心 | A kind of task processing method and device of sonde |
| CN108804211A (en) * | 2018-04-27 | 2018-11-13 | 西安华为技术有限公司 | Thread scheduling method, device, electronic equipment and storage medium |
| CN109308212A (en) * | 2017-07-26 | 2019-02-05 | 上海华为技术有限公司 | A kind of task processing method, task processor and task processing equipment |
| CN110119305A (en) * | 2019-05-13 | 2019-08-13 | 北京达佳互联信息技术有限公司 | Task executing method, device, computer equipment and storage medium |
| CN110489217A (en) * | 2019-07-11 | 2019-11-22 | 新华三大数据技术有限公司 | A kind of method for scheduling task and system |
| CN110837410A (en) * | 2019-10-30 | 2020-02-25 | 北京奇艺世纪科技有限公司 | Task scheduling method and device, electronic equipment and computer readable storage medium |
| CN110955502A (en) * | 2018-09-27 | 2020-04-03 | 北京国双科技有限公司 | Task scheduling method and device |
| CN110990142A (en) * | 2019-12-13 | 2020-04-10 | 上海智臻智能网络科技股份有限公司 | Concurrent task processing method and device, computer equipment and storage medium |
| CN111309372A (en) * | 2020-01-15 | 2020-06-19 | 中国平安财产保险股份有限公司 | Timed task execution method and device, computer equipment and storage medium |
-
2020
- 2020-06-23 CN CN202010579183.2A patent/CN111797110B/en active Active
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030225820A1 (en) * | 2002-05-31 | 2003-12-04 | Microsoft Corporation | System and method for collecting and storing event data from distributed transactional applications |
| US20120222035A1 (en) * | 2011-02-25 | 2012-08-30 | Qualcomm Incorporated | Priority Inheritance in Multithreaded Systems |
| US20130152098A1 (en) * | 2011-12-12 | 2013-06-13 | International Business Machines Corporation | Task priority boost management |
| US20170142027A1 (en) * | 2015-11-13 | 2017-05-18 | Raytheon Company | Dynamic priority calculator for priority based scheduling |
| CN105512027A (en) * | 2015-11-26 | 2016-04-20 | 珠海多玩信息技术有限公司 | Process state monitor method and device |
| CN107016480A (en) * | 2016-01-28 | 2017-08-04 | 五八同城信息技术有限公司 | Method for scheduling task, apparatus and system |
| CN107102894A (en) * | 2017-04-07 | 2017-08-29 | 百度在线网络技术（北京）有限公司 | Method for scheduling task, device and system |
| CN107391198A (en) * | 2017-07-25 | 2017-11-24 | 广东欧珀移动通信有限公司 | Method for scheduling task and device, computer-readable recording medium, mobile terminal |
| CN109308212A (en) * | 2017-07-26 | 2019-02-05 | 上海华为技术有限公司 | A kind of task processing method, task processor and task processing equipment |
| CN107977268A (en) * | 2017-10-13 | 2018-05-01 | 北京百度网讯科技有限公司 | Method for scheduling task, device and the computer-readable recording medium of the isomerization hardware of artificial intelligence |
| US20190114202A1 (en) * | 2017-10-13 | 2019-04-18 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Task scheduling method and apparatus of artificial intelligence heterogeneous hardware, device and readable medium |
| CN108132837A (en) * | 2018-01-02 | 2018-06-08 | 中国工商银行股份有限公司 | A kind of distributed type assemblies dispatch system and method |
| CN108717378A (en) * | 2018-04-20 | 2018-10-30 | 中国气象局气象探测中心 | A kind of task processing method and device of sonde |
| CN108804211A (en) * | 2018-04-27 | 2018-11-13 | 西安华为技术有限公司 | Thread scheduling method, device, electronic equipment and storage medium |
| CN110955502A (en) * | 2018-09-27 | 2020-04-03 | 北京国双科技有限公司 | Task scheduling method and device |
| CN110119305A (en) * | 2019-05-13 | 2019-08-13 | 北京达佳互联信息技术有限公司 | Task executing method, device, computer equipment and storage medium |
| CN110489217A (en) * | 2019-07-11 | 2019-11-22 | 新华三大数据技术有限公司 | A kind of method for scheduling task and system |
| CN110837410A (en) * | 2019-10-30 | 2020-02-25 | 北京奇艺世纪科技有限公司 | Task scheduling method and device, electronic equipment and computer readable storage medium |
| CN110990142A (en) * | 2019-12-13 | 2020-04-10 | 上海智臻智能网络科技股份有限公司 | Concurrent task processing method and device, computer equipment and storage medium |
| CN111309372A (en) * | 2020-01-15 | 2020-06-19 | 中国平安财产保险股份有限公司 | Timed task execution method and device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 曹宗雁;赵毅;牛铁;朱鹏;迟学斌;: "基于用户评价的集群作业优先级调度", 华中科技大学学报(自然科学版), no. 1, pages 139 - 143 * |
| 李磊;: "ETL任务集群调度方法", 计算机技术与发展, no. 11, pages 41 - 44 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111797110B (en) | 2024-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20190228303A1 (en) | Method and apparatus for scheduling resource for deep learning framework | |
| CN110765187A (en) | Data source route management method and device | |
| CN115335821B (en) | Offloading statistics collection | |
| CN110555068A (en) | Data export method and device | |
| CN112597126A (en) | Data migration method and device | |
| CN110716956A (en) | Data request intercepting method and device | |
| CN111062572A (en) | Task allocation method and device | |
| CN113190517A (en) | Data integration method and device, electronic equipment and computer readable medium | |
| CN110362583A (en) | A kind of data processing method and device for multi-data source | |
| CN109087097B (en) | Method and device for updating same identifier of chain code | |
| CN108810130B (en) | Method and device for planning distribution request | |
| CN112948138A (en) | Method and device for processing message | |
| CN112395337A (en) | Data export method and device | |
| CN111797110B (en) | Method, device, computer equipment and storage medium for generating scheduling model | |
| CN113626472A (en) | Method and device for processing order data | |
| CN113726885A (en) | Method and device for adjusting flow quota | |
| CN113722315A (en) | Data generation method and device, electronic equipment and computer readable medium | |
| CN113011858A (en) | Audit project configuration and execution method and device | |
| CN112463616A (en) | Chaos testing method and device for Kubernetes container platform | |
| CN112115154A (en) | Data processing and data query method, device, equipment and computer readable medium | |
| CN112507676A (en) | Energy report generation method and device, electronic equipment and computer readable medium | |
| CN111984616A (en) | Method, device and system for updating shared file | |
| CN111625692A (en) | Feature extraction method, device, electronic equipment and computer readable medium | |
| CN112988806A (en) | Data processing method and device | |
| CN110262756B (en) | Method and device for caching data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |