CN112514352A

CN112514352A - Method, device, system, storage medium and terminal for updating scheduling rule

Info

Publication number: CN112514352A
Application number: CN201880096166.4A
Authority: CN
Inventors: 李婧; 陈雪
Original assignee: Siemens AG
Current assignee: Siemens AG
Priority date: 2018-09-28
Filing date: 2018-09-28
Publication date: 2021-03-16
Also published as: WO2020062047A1

Abstract

The application relates to a method, a device, a system, a storage medium and a terminal for updating a scheduling rule. The method comprises the following steps: acquiring a work to be executed; acquiring a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in a system which needs to execute the work; distributing the operation to a device capable of executing the operation according to a scheduling rule; acquiring a system state of a system when equipment executes operation; generating a quality index from the system state, the quality index representing an assessment of the quality of the system performed work; generating training data according to the scheduling rule, the system state and the quality index; performing machine learning according to the training data to generate a meta rule, wherein the meta rule represents a scheduling rule to be adopted in different system states; and updating the scheduling rules in the set of scheduling rules according to the meta-rule. The method and the system have the advantages that the scheduling program is trained through the learning framework, the optimized scheduling rule is identified, and the complexity of developing the optimized scheduling program is reduced.

Description

Method, device, system, storage medium and terminal for updating scheduling rule

Technical Field

The present application relates to the field of scheduling control. In particular, the present application relates to a method, device, system, storage medium and terminal for updating a scheduling rule.

Background

Scheduling software is widely used in manufacturing environments. As shown in fig. 1, a scheduler 2 (also called scheduler or production manager) is responsible for analyzing jobs including the order 11, (manufacturing) process information 13, and warehouse information 15 of the job and process 1, dividing the job J into operation and scheduling information O & S, and then calculating an appropriate schedule of available resources 3 to perform the operations. For example, the resources 3 include a machine 31, a production line 33, a robot 35, an Automatic Guided Vehicle (AGV)37, and the like.

To implement an optimized scheduler for a manufacturing application, a number of algorithms have been developed. For example, optimization algorithms (heuristics) such as general purpose algorithms, simulated annealing, and tabu search were developed. However, these optimization algorithms are difficult to implement or adjust and are computationally too complex to be used in real-time systems. In addition, these algorithms are better suited to solve scheduling problems in static environments where the known amount of work and corresponding preparation time are fixed prior to actual scheduling execution. However, in practical applications, manufacturing often faces dynamically changing environmental issues, with work continuously appearing in the process of execution.

Disclosure of Invention

The embodiment of the application provides a method, equipment, a system, a storage medium and a terminal for updating a scheduling rule, so as to at least solve the problem that the scheduling rule is difficult to optimize in a dynamically changing environment in the prior art.

According to an aspect of an embodiment of the present application, there is provided a method for updating a scheduling rule, including: acquiring a work to be executed; acquiring a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in a system which needs to execute the work; distributing the operation to a device capable of executing the operation according to a scheduling rule; acquiring a system state of a system when equipment executes operation; generating a quality index from the system state, the quality index representing an assessment of the quality of the system performed work; generating training data according to the scheduling rule, the system state and the quality index; performing machine learning according to the training data to generate a meta rule, wherein the meta rule represents a scheduling rule to be adopted in different system states; and updating the scheduling rules in the set of scheduling rules according to the meta-rule.

In this way, the work is distributed according to the existing scheduling rules, the equipment in the system executes the work operation, the system state is obtained for evaluating the quality of the work executed according to the existing scheduling rules, the training data is generated, the influence of various existing scheduling rules and the corresponding system state on the quality of the work executed is obtained, the existing scheduling rules are updated, and the scheduling based on the scheduling rules can be dynamically adjusted according to the environment.

According to an exemplary embodiment of the application, the method further comprises, before obtaining the work to be performed: acquiring a work type, wherein the work type at least represents operations included in the work, equipment capable of executing the operations in the system and time for the equipment to execute the operations; and generating at least one job of the job type to be executed according to the job type.

In this way, a large amount of work can be generated for acquiring training data based on the scheduling rules.

According to an exemplary embodiment of the application, generating training data comprises: determining whether the quality index is less than a threshold value by comparing the quality index with a preset threshold value; and if the quality index is smaller than the threshold, generating a data table as training data according to the scheduling rule corresponding to the quality index and the system state, wherein the data table comprises the system state, the variable of the system state recorded along with time and the corresponding scheduling rule.

In this way, training data that can be effectively used to improve scheduling performance among all data is selected.

According to an exemplary embodiment of the application, machine learning from training data comprises: and determining an implicit relation between the quality of system execution work and the system state according to a machine learning algorithm and training data, wherein the implicit relation is used for generating meta-rules.

In this way, learning what system states will correspond to what quality of work or scheduling performance is the basis for optimizing the scheduling rules.

According to an exemplary embodiment of the application, updating the scheduling rules in the set of scheduling rules comprises: generating an update rule according to the meta-rule, wherein the update rule represents a scheduling rule to be adopted in a specific system state; and applying the update rule in conjunction with the scheduling rule to the distribution of the operation.

In this way, the optimized scheduling rule is generated according to the machine learning result, so that the updating rule capable of being dynamically scheduled according to the environment is included in the scheduling rule.

According to an exemplary embodiment of the application, distributing an operation to a device capable of performing the operation comprises: generating a distribution sequence of distribution operation according to a scheduling rule; and distributing operations according to the distribution sequence.

In this manner, operations are distributed to devices capable of performing the operations to perform work in the system.

According to an exemplary embodiment of the present application, the quality index is a weighted value of a plurality of attributes related to the quality of performing the work.

In this way, the scheduling performance is analyzed on the basis of a plurality of quality-affecting parameters.

According to an exemplary embodiment of the application, the method further comprises: after the training data is generated, the training data is stored in a database.

In this manner, the training data is used for subsequent machine learning or other uses.

According to another aspect of the embodiments of the present application, there is also provided an apparatus for updating a scheduling rule, including: a scheduling unit configured to: acquiring a work to be executed; acquiring a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in a system which needs to execute the work; distributing the operation to a device capable of executing the operation according to a scheduling rule; a training unit, the training unit comprising: a predictor attribute module configured to obtain a system state of the system when the device performs an operation; a quality assessment module configured to generate a quality index from the system state, the quality index representing an assessment of a quality of work performed by the system; and a data conversion module configured to generate training data according to the scheduling rules, the system state and the quality index; and a learning unit configured to: performing machine learning according to the training data to generate a meta rule, wherein the meta rule represents a scheduling rule to be adopted in different system states; and updating the scheduling rules in the set of scheduling rules according to the meta-rule.

In this way, according to the existing scheduling rule, the work is distributed, the equipment in the system executes the work operation, the system state is obtained for evaluating the quality of the work executed according to the existing scheduling rule, the training data is generated, so that the influence of various existing scheduling rules and the corresponding system state on the quality of the work executed is obtained, the existing scheduling rules are updated, and the scheduling of the scheduling unit based on the scheduling rules can be dynamically adjusted according to the environment.

According to an exemplary embodiment of the application, the apparatus further comprises: a job generation unit configured to: the method comprises the steps of obtaining a work type, wherein the work type at least represents operations included in the work, equipment capable of executing the operations in the system and time for the equipment to execute the operations, and generating at least one work of the work type to be executed according to the work type.

In this way, a lot of work can be generated for the scheduling unit for obtaining training data based on the scheduling rules.

According to an exemplary embodiment of the application, the apparatus further comprises: a database configured to store training data.

According to another aspect of the embodiments of the present application, there is also provided a system for updating a scheduling rule, including: a work system to perform a work, the work system including a device for performing an operation of the work; and a device for updating the scheduling rules, the device comprising: a scheduling unit configured to: acquiring a work to be executed; acquiring a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in a work system; distributing the operation to a device capable of executing the operation according to a scheduling rule; a training unit, the training unit comprising: the predictor attribute module is configured to acquire the system state of the working system when the equipment performs operation; a quality assessment module configured to generate a quality index from the system state, the quality index representing an assessment of quality of performing work on the work system; and a data conversion module configured to generate training data according to the scheduling rules, the system state and the quality index; and a learning unit configured to: performing machine learning according to the training data to generate a meta rule, wherein the meta rule represents a scheduling rule to be adopted in different system states; and updating the scheduling rules in the set of scheduling rules according to the meta-rule.

According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program, wherein when the program runs, a device on which the storage medium is located is controlled to execute the method of any one of the above.

According to another aspect of the embodiments of the present application, there is also provided a processor configured to execute a program, where the program executes to perform the method of any one of the above.

According to another aspect of the embodiments of the present application, there is also provided a terminal, including: one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the method of any of the above.

According to another aspect of embodiments of the present application, there is also provided a computer program product, tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method of any one of the above.

In this way, the technical solution according to the present application can be implemented on a computer in a software and program manner, optimizing the scheduling rules.

The rule-based scheduling algorithm aims at selecting the next job to be processed from the jobs waiting to be served in the queue. They can handle dynamic environments and are relatively easy to implement. Simulation has proven to be an important strategic tool for evaluating different rules applied to a scheduler in order to find optimized rules for a simulation scenario. The method for establishing the systematic self-learning scheduling rule in the simulation tool can be used. In addition, the technical scheme of the application can also be applied in a real scene so as to update and optimize the scheduling rule according to the operation of a real system and equipment. Moreover, the present application is also capable of generating a large amount of training data regarding the performance of rule-based scheduling decisions and their corresponding system states. In the present application, an implicit relationship between scheduling performance and system state (represented by predictor attributes) is explored by adopting a machine learning method. The discovered implicit relationship becomes a "meta-rule" for scheduling, and the scheduling decision can be dynamically adjusted according to the system state by using the rule, for example, an appropriate scheduling rule is selected according to the system state. Thus, the scheduler is trained to dynamically adjust the scheduling rules according to the system state.

In the embodiment of the application, a technical scheme for updating the scheduling rule according to the relationship between the system state and the scheduling performance learned by the training data based on the scheduling rule is provided, so that the technical problem that the optimized scheduling rule is difficult to find in the dynamic environment is at least solved, and the technical effect of efficiently realizing work distribution in the dynamic environment is realized.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic diagram of a scheduler distributing work to resources;

FIG. 2 is a flow chart of a method of updating scheduling rules according to an embodiment of the present application;

FIG. 3 is a flow chart of a method of generating a job in accordance with an exemplary embodiment of the present application;

FIG. 4 is a flowchart of a method of generating training data according to an exemplary embodiment of the present application;

FIG. 5 is a schematic diagram of an exemplary decision tree of a method of updating scheduling rules according to an embodiment of the present application;

FIG. 6 is a block diagram of an apparatus for updating scheduling rules according to an embodiment of the present application;

fig. 7 is a block diagram of an apparatus for updating a scheduling rule according to an exemplary embodiment of the present application;

FIG. 8 is a block diagram of a system for updating scheduling rules according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a dispense operation according to an embodiment of the present application.

The reference numbers illustrate:

1, working and processing;

11, ordering;

13, process information;

15, warehouse information;

j, the operation is carried out,

the O & S, operation and scheduling information,

2, a scheduler;

3, resources;

31, a machine;

33, a production line;

35, a robot;

37, automatically guiding the transport vehicle;

s201 to S215: a step of;

s301 to S303: a step of;

s401 to S403: a step of;

X1-X7: a state;

R1-Rm: a rule;

6, equipment for updating the scheduling rules;

61, a scheduling unit;

63, a training unit;

631, predictor attribute module;

633, a quality evaluation module;

635, a data conversion module;

65, a learning unit;

67, a job generation unit;

69, a database;

8, updating a system of scheduling rules;

10, a working system;

101, a device for performing operations of a job;

M1-M7 as devices of resources

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules or elements is not necessarily limited to those steps or modules or elements expressly listed, but may include other steps or modules or elements not expressly listed or inherent to such process, method, article, or apparatus.

Some simulation tools include optimization algorithms, such as linear/integer programming or metaheuristic algorithms, such as genetic algorithms and tabu search, to solve scheduling problems. However, the amount of work in a static environment that these algorithms handle is known, which makes it difficult to meet the dynamic requirements of a real situation. In practical situations, work may be constantly being manifested during execution. Furthermore, optimization solutions are often difficult to implement and optimize, and are computationally too complex to use in real-time systems. In contrast, rule-based scheduling prioritizes the pending jobs using predefined scheduling rules, such as priority levels. They are easy to implement and greatly reduce the computational requirements, but their scheduling performance is generally low.

Implicit scheduling rules can be discovered from optimization or meta-heuristic algorithms using machine learning methods. The learning process comprises the following steps: a) running an optimization algorithm to calculate an optimal scheduling decision; b) finding a preferred scheduling decision using simulation; c) a machine learning algorithm is applied to retrieve implicit knowledge. These methods have problems in that; 1) only the job-shop scheduling problem (JSSP) is concerned, which is only a specific scheduling problem (sub-scheduling) compared to the technical framework according to the present application; 2) generating scheduling by relying on an optimization algorithm, and realizing and configuring the optimization algorithm for each training example with high complexity, 3) manually constructing predictor attributes; 4) there is a lack of systematic simulation tools to learn process automation methods. The application provides a method for learning meta-rules, and explores the relationship between the scheduling performance of various scheduling rules and the system state of the scheduling rules.

In addition, there is no systematic way in simulation to extract implicit knowledge rules from simulation cases to continuously improve the scheduler. Typically, the scheduler relies on the expertise of an engineer to manually write code that simulates the plant case. Existing simulation tools do not have embedded frameworks that can automatically accumulate and utilize simulation result data for knowledge learning to improve scheduling performance.

According to an embodiment of the present application, a method of updating a scheduling rule is provided. Fig. 2 is a flowchart of a method of updating a scheduling rule according to an embodiment of the present application. As shown in fig. 2, a method for updating a scheduling rule according to an embodiment of the present application is described in conjunction with the following.

In step S201, a job to be executed is acquired. The job is a job executed by the system, for example, a job generated from an order received by the production system, or a job generated in the simulation system. In an embodiment according to the application, the scheduling performance of a distributed job when a real system or a simulation system performs the job is analyzed. The work obtained will be assigned to the production devices in the field or mapped to the simulation devices in the simulation system, depending on whether the data on the schedule is obtained from the field or from the simulation devices using simulation techniques in the use of the method.

Next, in step S203, a scheduling rule set including a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in the system that is to execute the work is acquired. The set of scheduling rules is pre-stored or may be obtained or determined based on input. For example, the scheduling rule set includes a plurality of scheduling rules, such as a priority rule and a random rule, and may also include other rules for scheduling. The scheduling rule indicates that, for a specific individual operation among a plurality of operations included in a certain job, the operation should be executed by a corresponding device capable of executing the operation under some specific conditions.

Next, in step S205, an operation is distributed to devices capable of performing the operation according to the scheduling rule. If the operation of the work is performed by the field device, the operation is distributed to the field device, and the actual operation is performed. If a simulation system is employed, work is distributed to simulation devices, such as simulation devices to which field devices are mapped in the simulation system, to perform the simulation process.

Next, in step S207, the system state of the system at the time when the device performs the operation is acquired. A real system may generate a corresponding system state when its device performs an operation, or a simulated system may generate predictor attributes to represent the system state when it performs a simulated operation. And acquiring the system state of the equipment during operation, and providing training data for a subsequent algorithm. For example, the system state may be recorded at certain time intervals during the performance of the operation, or the system state may be recorded at a particular time. And for the simulation process, recording the system state in the simulation system when the simulation equipment executes the operation after the operation is distributed to the simulation equipment. The system status is, for example and without limitation, one or more of the number of jobs outstanding in the system, the current load of the device of interest, the percentage of jobs relatively close to the completion date, the average remaining time to the due date, the percentage of jobs requiring relatively long processing time, the difference between the maximum processing time and the average processing time, the difference in processing time of the two operations, and the like. These system states reflect the system attributes or scheduling capabilities of interest and may be selected and set according to actual needs.

For example, an exemplary system state may be represented by the following attributes:

x1: the current load on the device of interest;

x2: the number of outstanding jobs in the system;

x3: the percentage of jobs that should be completed with a relatively close date;

x4: average remaining time to due date;

x5: percentage of jobs with relatively long processing times;

x6: a difference between the maximum remaining processing time and the average remaining processing time;

x7: the processing time difference of the two operations to be compared.

……

Data for these attributes is recorded over time to continuously represent the system state. For different analysis purposes, different attributes of the system state may be selected to be better used for updating the scheduling rules.

Next, in step S209, a quality index is generated from the system state, the quality index representing an evaluation of the quality of the system performing work. The quality of the work reflects the quality of the work performed by the system, and can represent the quality of the scheduling decision made according to the corresponding scheduling rule distribution operation, such as, but not limited to, the efficiency after distributing the operation of the work to a specific device according to a specific scheduling rule for execution, the system resource occupancy rate, the preparation time or the average duration of the work, and the like. The quality of the job is determined based on the scheduling objective, e.g., if the speed at which the system performs the job is a concern, the quality of the job is considered as the preparation time and duration of the job, and the quality is evaluated to generate a quality index.

Next, in step S211, training data is generated according to the scheduling rule, the system state, and the quality index. For the scheduling rule, training data is established after the corresponding system state and quality index are acquired, which represents the system state recorded over time and the evaluation of the quality of the work performed according to the scheduling rule (the evaluation of scheduling performance) after the operation is distributed to the equipment according to the specific scheduling rule.

Next, in step S213, machine learning is performed based on the training data to generate a meta rule indicating a scheduling rule to be adopted in a different system state. Machine learning is performed using the training data generated in step S211 to explore the relationship between the scheduling rules and the system states, and further analysis is performed to generate meta-rules from the relationship, which indicate in which system states which scheduling rules should be employed to achieve a particular scheduling objective. For example, a set of existing data mining and machine learning algorithms, such as decision tree C5.0, Artificial Neural Networks (ANNs), may be inserted to learn patterns and implicit knowledge from the accumulated simulation data. The goal to learn is to determine which scheduling rule is more appropriate in a particular system state. Extracting this knowledge from the training data will allow meta-rules to be generated, selecting different scheduling rules for any set of jobs at any given time. Fig. 5 is a schematic diagram of an exemplary decision tree of a method of updating a scheduling rule according to an embodiment of the present application. As shown in fig. 5, implicit knowledge is obtained from the simulation results using a decision tree C5.0 as a learning mechanism. By applying the C5.0 algorithm to the training data, knowledge of meta-rules as shown in fig. 5 can be finally obtained, e.g., X1-X7 represent states in the system, according to which the meta-rules can direct the scheduler to apply the scheduling rules R1-Rm according to the states. For example, if the correlation value of state X1 is less than or equal to threshold A, the system enters state X2; if the correlation value of X1 is greater than the threshold A, the system maintains state X1. Next, if the correlation value of state X2 is less than or equal to the threshold B1, the system enters state X3 and the scheduler applies the scheduling rule R1. If the correlation value of X1 is within the interval of the thresholds B1 and B2, the system enters state X5 and the scheduler applies rule R2. If the correlation value of X1 is greater than the threshold B2, then a scheduling decision is made by the scheduler, and the system enters state X6 after the scheduling decision is made, and the scheduler will apply the corresponding rule. As the above only illustrates the way to direct the scheduler to apply the rules, the different state orders correspond to different rules R1-Rm, so that the rule set of the scheduler can be continuously improved.

Next, in step S215, the scheduling rules in the scheduling rule set are updated according to the meta-rule. Through the above steps, the predetermined scheduling rule is updated to obtain a new scheduling rule, or the original scheduling rule is modified for the subsequent scheduling process of work distribution, or the scheduling rule is further updated to obtain a more optimized scheduling rule. In the method of updating a scheduling rule according to the present application, the data for updating the scheduling rule employs a predicted value, a quality index value, and a rule obtained at each recording. In this way, the work is distributed according to the existing scheduling rules, the equipment in the system executes the work operation, the system state is obtained for evaluating the quality of the work executed according to the existing scheduling rules, the training data is generated, the influence of various existing scheduling rules and the corresponding system state on the quality of the work executed is obtained, the existing scheduling rules are updated, and the scheduling based on the scheduling rules can be dynamically adjusted according to the environment.

A method of generating a job is provided according to an exemplary embodiment of the present application. FIG. 3 is a flowchart of a method of generating a job according to an exemplary embodiment of the present application. As shown in fig. 3, a method for generating a job according to an exemplary embodiment of the present application includes performing step S301 before acquiring a job to be performed, acquiring a job type, where the job type indicates at least an operation included in the job, a device capable of performing the operation in the system, and a time when the device performs the operation. Then, step S303 is performed to generate at least one job of the job type to be executed according to the job type. The method of generating work according to an exemplary embodiment of the present application is performed before the method of updating the scheduling rules according to fig. 2, and when the method of updating the scheduling rules is performed based on a simulation system, a large amount of data of work may be generated for the simulation system for generating training data. For a dynamic work arrival process, the arrival events may follow a probability distribution, such as a uniform distribution or a poisson distribution. An example of the work is shown in table 1:

work 1	Work 2
The working type is as follows: production of X	The working type is as follows: assembly Y
Quantity: 10	Quantity: 5
Arrival time: t1	Arrival time: t2
Expiration time: t3	Expiration time: t4
Priority: 0	Priority: 1
……	……

TABLE 1

Job 1 is based on "job type: production X ", job 2 is according to" job type: assembly Y ". Job 1 and job 2 are sent to the device to perform the corresponding operations. The working type is as follows: examples of production of X are as follows:

the process of the dispensing operation is shown according to an embodiment of the application. FIG. 9 is a schematic diagram of a dispense operation according to an embodiment of the present application. As shown in fig. 9, in this example, the sequence of operations includes operation 1, operation 2, operation 3, and operation 5, for example, the work type: production X ═ operation 1, operation 2, operation 3, operation 5 }. The sequence of operations shown here is merely illustrative, and the series of operations may include more operations or fewer operations, depending on how many operations are included in the job.

The resources shown in this example for performing operations include M1, M2, M3, M4, M5, M6, and M7. The resources may include more or fewer devices, depending on the device conditions. In this example, operation 1 can be performed by devices M1, M3, and M6, performing processes P11, P13, and P16, respectively, and performing

operations

2 and 3 after completion; operation 2 can be performed by devices M2 and M4, performing processes P22 and P24, respectively, and performing operations 3 and 5 after completion; operation 3 can be performed by devices M1, M3, and M6, performing processes P31, P33, and P36, respectively, and performing operation 5 after completion; operation 5 can be performed by devices M3 and M7, performing processes P53 and P57, respectively, and mapping of operations to devices as shown in the above example.

A series of jobs as described above can be generated according to the job type as described above and distributed to the simulation devices. The simulation device can execute a simulation process to obtain a simulated processing result.

A method of generating training data is provided according to an exemplary embodiment of the present application. Fig. 4 is a flowchart of a method of generating training data according to an exemplary embodiment of the present application. As shown in fig. 4, generating training data according to an exemplary embodiment of the present application includes: step S401, comparing the quality index with a preset threshold value, and determining whether the quality index is smaller than the threshold value. A threshold is predetermined for distinguishing between data intended for use in generating training data and data not considered for use in generating training data. Step S403 is performed, and if the quality index is smaller than the threshold, a data table is generated as training data according to the scheduling rule and the system state corresponding to the quality index, where the data table includes the system state, the variables of the system state recorded over time, and the corresponding scheduling rule. An arrangement may be made to select data for which the quality index is less than a threshold value to generate training data. For example, for the quality of the work "shortest time to complete the task", the quality index represents "time to complete", which is less than a predetermined threshold, indicating that the work is performed more efficiently according to the rule, and therefore the corresponding data is used to generate training data to obtain the optimized scheduling rule. It should be understood that the predetermined threshold is determined according to the evaluation of the scheduling performance concerned, if the system resource utilization rate is concerned, the "average duration of resource idle time" may be used as the quality index, and when the quality index is smaller than the threshold, it indicates that the system resource utilization rate is high, the quality of the scheduling decision is good, and the corresponding data can be used to generate the training data according to the corresponding rule. Depending on what the specifically selected quality index represents, the quality index may also be larger than a threshold, with the aim of determining the quality of the scheduling decision by comparing the quality index with the threshold, e.g. the quality of the scheduling decision is higher than a preset threshold or a desired quality. The training data comprises a scheduling rule and a system state adopting the scheduling rule, and the data is recorded in a data table form so as to clearly represent the system state, variables of the system state recorded along with time and the corresponding scheduling rule, thereby facilitating the processing of a machine learning process.

Table 2 is an exemplary data table, with attribute X being the predictor attribute of the simulation system representing the system state:

training data

At time 0

At time 1

…

At time x

Collection	Value of	Value of		Value of
Attribute X1 >	0.5	0.8	…	0.2
Attribute X2 >	3	2	…	1
Attribute X3 >	0.1	0.3	…	0.1
Attribute X4 >	30.	10	…	3
…	…	…	…	…
Attribute X7 >	4	0	…	10
Rule #	r ₁	r ₂	…	r ₁

TABLE 2

According to an exemplary embodiment of the application, after obtaining valid training data, performing machine learning according to the training data includes determining an implicit relationship between a quality of system execution work and a system state according to a machine learning algorithm and the training data, the implicit relationship being used to generate meta-rules. For example, in step S213, the training data is processed by using a machine learning algorithm to obtain implicit knowledge, such as using a specific scheduling rule in a specific system state. In this way, learning what system states will correspond to what quality of work or scheduling performance is the basis for optimizing the scheduling rules.

According to an exemplary embodiment of the application, after obtaining the meta-rule, updating the scheduling rule in the set of scheduling rules comprises generating the update rule according to the meta-rule, the update rule representing the scheduling rule to be employed in a specific system state, and applying the update rule together with the scheduling rule to the distribution of the operation. In this way, the optimized scheduling rule is generated according to the machine learning result, so that the updating rule capable of being dynamically scheduled according to the environment is included in the scheduling rule.

According to an exemplary embodiment of the application, distributing an operation to a device capable of performing the operation comprises: generating a distribution sequence of distribution operation according to a scheduling rule; and distributing operations according to the distribution sequence. The device on which the operation is to be performed and the order of the operations to be performed are determined according to the scheduling rules, in such a way that the operation is distributed to the devices capable of performing the operation to perform work in the system.

According to an exemplary embodiment of the present application, the quality index is a weighted value of a plurality of attributes related to the quality of performing the work. If multiple attributes need to be considered for the scheduling objective, then the importance of the preselected attributes is quantified in a weighted manner. Such as the quality index may be a weighted function of a plurality of objective functions. For example, an exemplary objective function f₁And f₂The method comprises the following steps:

f ₁: average duration of work lead timeTime

f ₂: average duration of resource idle time

The weighted quality index function may be: f ═ w₁f ₁+w ₂f ₂，w ₁And w₂Is predetermined according to the importance of the attribute to be considered. Downstream decisions may significantly affect the calculated quality index and therefore this factor should be taken into account when evaluating the quality index. The final quality index represents the scheduling performance considering a plurality of scheduling objectives. In this manner, scheduling performance is analyzed based on a plurality of quality-affecting attributes.

According to an exemplary embodiment of the application, the method further comprises: after the training data is generated, the training data is stored in a database. For example, training data is recorded in a database in the form { predictor attributes, rules }, and the module performing machine learning can call the data therein. In this manner, the training data is used for subsequent machine learning or other uses.

According to the embodiment of the application, equipment for updating the scheduling rule is further provided. Fig. 6 is a block diagram of an apparatus for updating a scheduling rule according to an embodiment of the present application. As shown in fig. 6, the apparatus 6 for updating a scheduling rule according to an embodiment of the present application includes: a scheduling unit 61 configured to: acquiring a work to be executed; acquiring a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in a system which needs to execute the work; distributing the operation to a device capable of executing the operation according to a scheduling rule; a training unit 63 comprising: a predictor attribute module 631 configured to obtain a system state of the system when the device performs an operation; a quality assessment module 633 configured to generate a quality index representing an assessment of the quality of the system performing work, in accordance with the system state; and a data transformation module 635 configured to generate training data according to the scheduling rules, the system state, and the quality index; and a learning unit 65 configured to: performing machine learning according to the training data to generate a meta rule, wherein the meta rule represents a scheduling rule to be adopted in different system states; and updating the scheduling rules in the set of scheduling rules according to the meta-rule. The device 6 for updating the scheduling rules according to an embodiment of the present application may be embedded in a simulation tool for running in a simulation system. The training unit 63 is responsible for recording the relevant system states in the simulation, building training data for subsequent learning algorithms, and the predictor attribute module 631 calculates a set of variables representing each system state. The quality evaluation module 633 evaluates the obtained system state after each scheduling decision. Finally, the data conversion module 635 further converts the collected simulation data into suitable training data as input to the subsequent learning unit 65. The learning unit 65 exploits implicit knowledge, i.e. implicit relations between scheduling performance and system state (expressed in predictor attribute variables), from the accumulated training data using a set of existing machine learning algorithms. The found "meta-rules" will be provided to the scheduling unit 61, using which the scheduling unit 61 will dynamically customize the scheduling decisions according to the system state and outperform conventional scheduling rules. The device 6 for updating the scheduling rule according to the embodiment of the present application executes the method for updating the scheduling rule according to the embodiment of the present application, and details thereof are not repeated.

In this way, the work is distributed according to the existing scheduling rules, the equipment in the system executes the work operation, the system state is obtained for evaluating the quality of the work executed according to the existing scheduling rules, the training data is generated, the influence of various existing scheduling rules and the corresponding system state on the quality of the work executed is obtained, and the existing scheduling rules are updated, so that the scheduling of the scheduling unit based on the scheduling rules can be dynamically adjusted according to the environment.

Fig. 7 is a block diagram of an apparatus for updating a scheduling rule according to an exemplary embodiment of the present application. As shown in fig. 7, according to an exemplary embodiment of the present application, the apparatus (6) further comprises: a job generation unit 67 configured to: the job type is acquired, the job type indicates at least an operation included in the job, a device capable of executing the operation in the system, and a time at which the device executes the operation, and at least one job of the job type to be executed is generated according to the job type, and a large number of jobs can be generated for the scheduling unit 61 for acquiring the training data based on the scheduling rule.

As shown in fig. 7, according to an exemplary embodiment of the present application, the apparatus (6) further comprises: a database (69) configured to store training data for subsequent machine learning or other use.

According to another aspect of the embodiments of the present application, a system for updating a scheduling rule is also provided. Fig. 8 is a block diagram of a system for updating scheduling rules according to an embodiment of the present application. As shown in fig. 8, a system 8 for updating a scheduling rule according to an embodiment of the present application includes: a work system 10 to perform work, the work system 10 including an apparatus 101 for performing an operation of the work; and a device 6 for updating the scheduling rules, the device 6 comprising: a scheduling unit 61 configured to: acquiring a work to be executed; acquiring a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in a work system; distributing the operation to a device capable of executing the operation according to a scheduling rule; a training unit 63, the training unit 63 comprising: a predictor attribute module 631 configured to obtain a system state of the work system when the device performs an operation; a quality assessment module 633 configured to generate a quality index representing an assessment of the quality of performing work on the work system according to the system state; and a data transformation module 635 configured to generate training data according to the scheduling rules, the system state, and the quality index; and a learning unit 65 configured to: performing machine learning according to the training data to generate a meta rule, wherein the meta rule represents a scheduling rule to be adopted in different system states; and updating the scheduling rules in the set of scheduling rules according to the meta-rule. In this way, the work is distributed according to the existing scheduling rules, the equipment in the system executes the work operation, the system state is obtained for evaluating the quality of the work executed according to the existing scheduling rules, the training data is generated, the influence of various existing scheduling rules and the corresponding system state on the quality of the work executed is obtained, the existing scheduling rules are updated, and the scheduling based on the scheduling rules can be dynamically adjusted according to the environment. The operation executed by the system for updating the scheduling rule in the embodiment of the present application refers to the above contents, and is not described herein again.

It should be understood that the technical solution of the present application can be implemented in real systems and simulation systems. When implemented in a real system, historical data of the performed operations and system states may be obtained from the field or a database, whereas in a simulated system, data required for training may be obtained through a simulation process.

The present application may also be implemented in the form of a program, and according to an embodiment of the present application, there is also provided a storage medium including the stored program, wherein, when the program runs, a device on which the storage medium is located is controlled to execute the above method. According to an embodiment of the present application, there is also provided a processor configured to execute a program, where the program executes the method. According to an embodiment of the present application, there is also provided a terminal including: one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the above-described method. There is also provided, in accordance with an embodiment of the present application, a computer program product, tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the above-described method. In this way, the technical solution according to the present application can be implemented on a computer in a software and program manner, optimizing the scheduling rules.

Through knowledge learned from a large amount of simulation data, the scheduling unit can make scheduling decisions more efficiently. The meta-rule will direct the scheduling unit to apply the most appropriate scheduling rule according to the system state. The dispatching unit in the trained simulation system calculates the system state through a predictor, and dynamically selects the most suitable dispatching rule according to the system state. The training process may continue in a frame with more data to determine the optimized scheduling rules.

The present application uses a learning structure in the simulation tool to enable the discovered knowledge to be used to continuously improve the performance of the scheduler. The embedded learning framework can train a scheduling program and identify optimized scheduling rules through simulation data. The system creates a new value proposition for the existing simulation tool, and reduces the obstacles and manpower for developing the high-level scheduling program.

According to the method and the system, fewer new modules are built in the simulation tool, and the automatic collection, construction and storage system of training data in the knowledge learning process is realized. The system may also implement new business models in the future, such as sharing simulation results, to train different schedulers.

In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units or modules is only one logical division, and there may be other divisions when the actual implementation is performed, for example, a plurality of units or modules or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of modules or units through some interfaces, and may be in an electrical or other form.

The units or modules described as separate parts may or may not be physically separate, and parts displayed as units or modules may or may not be physical units or modules, may be located in one place, or may be distributed on a plurality of network units or modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional units or modules in the embodiments of the present application may be integrated into one processing unit or module, or each unit or module may exist alone physically, or two or more units or modules may be integrated into one unit or module. The integrated unit or module may be implemented in the form of hardware, or may be implemented in the form of a software functional unit or module.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

A method for updating a scheduling rule, comprising:

acquiring a work to be executed;

obtaining a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in a system for executing the work;

distributing the operation to a device capable of executing the operation according to the scheduling rule;

acquiring a system state of the system when the equipment executes the operation;

generating a quality index from the system state, the quality index representing an assessment of a quality of the system performing the work;

generating training data according to the scheduling rule, the system state and the quality index;

performing machine learning according to the training data to generate a meta-rule, wherein the meta-rule represents a scheduling rule to be adopted in different system states; and

and updating the dispatching rules in the dispatching rule set according to the meta-rule.
The method of claim 1, wherein prior to obtaining the work to be performed:

acquiring a work type, wherein the work type at least represents an operation included in the work, equipment capable of executing the operation in the system and time for the equipment to execute the operation; and

generating at least one job of the job type to be executed according to the job type.
The method of claim 1 or 2, wherein generating training data comprises:

determining whether the quality index is less than a preset threshold value by comparing the quality index with the threshold value;

and if the quality index is smaller than the threshold, generating a data table as the training data according to the scheduling rule corresponding to the quality index and the system state, wherein the data table comprises the system state, the variable of the system state recorded along with time and the corresponding scheduling rule.
The method of claim 1 or 2, wherein machine learning from the training data comprises:

determining an implicit relationship between the quality of the system performing the work and the system state from a machine learning algorithm and the training data, the implicit relationship being used to generate the meta-rule.
The method of claim 1 or 2, wherein updating the scheduling rules in the set of scheduling rules comprises:

generating an update rule according to the meta-rule, wherein the update rule represents the scheduling rule to be adopted in a specific system state; and

applying the update rule in conjunction with the scheduling rule to the distribution of the operation.
The method of claim 1 or 2, wherein distributing the operation to a device capable of performing the operation comprises:

generating a distribution sequence for distributing the operation according to the scheduling rule; and

distributing the operations according to the distribution sequence.
The method of claim 1 or 2, wherein the quality index is a weighted value of a plurality of attributes related to the quality of performing the work.
The method of claim 1 or 2, further comprising:

after generating the training data, storing the training data in a database.
An apparatus for updating a scheduling rule, comprising:

a scheduling unit (61) configured to:

acquiring a work to be executed;

obtaining a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing operations included in the work to devices capable of executing the operations in a system for executing the work;

distributing the operation to a device capable of executing the operation according to the scheduling rule;

a training unit (63) comprising:

a predictor attribute module (631) configured to obtain a system state of the system while the device is performing the operation;

a quality assessment module (633) configured to generate a quality index representing an assessment of a quality of the system performing the work, in dependence on the system state; and

a data conversion module (635) configured to generate training data according to the scheduling rules, the system states, and the quality index; and

a learning unit (65) configured to:

performing machine learning according to the training data to generate a meta-rule, wherein the meta-rule represents a scheduling rule to be adopted in different system states; and

and updating the dispatching rules in the dispatching rule set according to the meta-rule.
The apparatus of claim 9, further comprising:

a job generation unit (67) configured to:

obtaining a job type representing at least an operation included in the job, a device capable of performing the operation in the system, and a time at which the device performs the operation, an

Generating at least one job of the job type to be executed according to the job type.
The apparatus of claim 9, further comprising:

a database (69) configured to store the training data.
A system for updating scheduling rules, comprising:

a work system (10) to perform work, the work system (10) comprising a device (101) for performing operations of the work; and

device (6) for updating a scheduling rule, the device (6) comprising:

a scheduling unit (61) configured to:

acquiring a work to be executed;

obtaining a scheduling rule set, wherein the scheduling rule set comprises a plurality of scheduling rules for distributing the operation included in the work to equipment capable of executing the operation in the work system;

distributing the operation to a device capable of executing the operation according to the scheduling rule;

a training unit (63) comprising:

a predictor attribute module (631) configured to obtain a system state of the work system while the device is performing the operation;

a quality assessment module (633) configured to generate a quality index representing an assessment of a quality of the work performed by the work system in dependence on the system state; and

a data conversion module (635) configured to generate training data according to the scheduling rules, the system states, and the quality index; and

a learning unit (65) configured to:

performing machine learning according to the training data to generate a meta-rule, wherein the meta-rule represents a scheduling rule to be adopted in different system states; and

and updating the dispatching rules in the dispatching rule set according to the meta-rule.
Storage medium, characterized in that the storage medium comprises a stored program, wherein the program, when executed, controls an apparatus in which the storage medium is located to perform the method of any of claims 1 to 8.
Processor, characterized in that the processor is configured to run a program, wherein the program when running performs the method of any of claims 1 to 8.
A terminal, comprising: one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the method of any of claims 1-8.
A computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method of any one of claims 1 to 8.