KR20240000926A - A neural network-based factory scheduling system independent of product type and number of types - Google Patents

Abstract

Scheduling of product types is performed in each process by learning a neural network for only N product types. N candidate types are selected from more than N product types, and only the process states of the selected N candidate types are extracted and input into the neural network. A neural network-based process scheduling system independent of product type and number of types, comprising: a simulation execution unit that simulates the factory workflow using a factory simulator; a candidate type selection unit that selects product type candidates (hereinafter referred to as candidate types) in a workflow state simulated by the factory simulator, and selects as many candidate types as the number of product types in the scheduling neural network; a product type mapping unit that maps the selected candidate type and the product type of the scheduling neural network; And, generate learning data of the scheduling neural network using the workflow state simulated by the factory simulator, select a candidate type through the candidate type selection unit, and select the candidate type selected through the product type mapping unit and the scheduling. By mapping the product type of the neural network, a configuration including a learning execution unit that generates learning data corresponding to the corresponding workflow state is prepared.
By using the above system, N candidate types are selected and applied to the neural network, and the neural network is shared even if the product types combined into N are different, so that even if the combined product type changes, the neural network can be used without redesigning/relearning the neural network. The learned neural network can be used as is.

Description

A neural network-based factory scheduling system independent of product type and number of types }

The present invention learns a process-specific neural network for scheduling each process in a factory workflow consisting of a series of processes, but simulates the factory workflow using a simulator and learns the neural network with simulated data, independent of the product type and number of types. It relates to a neural network-based process scheduling system.

In addition, in a process situation where the type of product to be produced is continuously changing, the present invention selects a predetermined number of high-priority product types and trains a neural network only for these, so that the final product type among the N high-priority product types in each process It relates to a neural network-based process scheduling system that is independent of product type and number of types, and produces scheduling results. In particular, the present invention selects N highest candidate types in a situation where there are more than N various product types and the number of product types changes each time scheduling is performed, and the N highest candidate types corresponding to the selected N candidate types are used for this purpose. Only the process state is extracted and input into the neural network.

In general, factory scheduling refers to the task of determining the work order of the processes necessary for processing a workpiece, the materials needed for each process, and the timing of production during the manufacturing process from raw materials or materials to the completion of the product.

In particular, factories that produce products are equipped with equipment that handles each process task, arranged in the work space for that process. The equipment can be configured to be supplied with lots to process specific tasks. In addition, transfer devices such as conveyors are installed between equipment or work spaces, so that the processed lot is moved to the next process when a specific process is completed by the equipment. In other words, one lot is produced as a finished product through a series of processes.

Additionally, in order to perform a specific process, multiple pieces of equipment with similar/same functions may be installed and the same or similar process tasks may be divided and processed. Scheduling the process or each task in such a manufacturing line is a very important issue for factory efficiency. Conventionally, most scheduling was done in a rule-based format according to each condition, but the performance evaluation of the scheduling results was ambiguous because the evaluation scale was not clear.

Additionally, recently, technologies for scheduling work by introducing artificial intelligence techniques into the manufacturing process have been proposed [Patent Document 1]. The above prior art used a machine learning algorithm called a genetic algorithm among artificial intelligence technologies, but limited the work of machine tools to scheduling. In addition, a technology applying a neural network learning method to processes of multiple facilities is also proposed [Patent Document 2]. However, the above prior art is a technology that finds the optimal control method in a given situation based on past data, and has a clear limitation in that it does not work without data accumulated in the past.

In order to solve the above problems, the present applicant proposes a technology for simulating the process using a factory simulator and learning a neural network for each process using the simulated data [Patent Document 3]. However, at this time, nodes corresponding to the number of product types are placed in the output of the artificial neural network, and among the output nodes, the node with the largest signal (or number) size is selected, and the lot corresponding to the output node is artificially The neural network becomes the product of choice.

For example, the artificial neural network in the case where there are currently two product types, such as product A and product B, is shown in Figure 1(a). As shown in Figure 1, two output nodes representing products A and B are set at the output of the artificial neural network. In addition, the input side of the artificial neural network is composed of nodes representing the process status of products A and B, and the number of these input nodes is also affected by the number of product conditions.

Therefore, if the number of product types changes in the factory workflow, the nodes of the artificial neural network must be changed and learning must start again from the beginning. For example, if a factory with two product types, A and B, needs to newly produce product type C, a different type of artificial neural network from the artificial neural network in Figure 1(a) must be redesigned and retrained. . In other words, it must be redesigned and retrained as an artificial neural network of the type shown in Figure 1(b).

The number of products produced in a typical factory cannot be defined in advance. Therefore, the above prior art has a problem in that the entire artificial neural network must be redesigned/relearned every time the number of products produced changes. In other words, AlphaGo's artificial neural network that makes decisions on the 19×19 Go problem has as many output nodes as 19×19, but when the Go board is changed to 20×20, the entire neural network must be redesigned and retrained.

Korean Patent Publication No. 10-1984460 (announced on May 30, 2019) Korean Patent Publication No. 10-2035389 (announced on October 23, 2019) Korean Patent Publication No. 10-2338304 (announced on December 13, 2021)

V. Mnih et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, p. 529, 2015. The Goal: A Process of Ongoing Improvement, Eliyahu M. Goldratt 1984

The purpose of the present invention is to solve the problems described above, and in an environment of factory production product types that are different for each factory and fluctuate even within a specific factory, each process is performed by learning a neural network only for a constant number of N product types. Perform scheduling of product types in . To this end, N candidate types are selected from N or more product types, and only the process states of the selected N candidate types are extracted and input into the neural network, providing a neural network-based process scheduling system independent of product type and number of types. will be.

In particular, the purpose of the present invention is to share the neural network of the process, learn the neural network, and apply it to the neural network for scheduling of the process, independent of the product type and number of types, even if the product types combined into N are different. It provides a neural network-based process scheduling system.

In order to achieve the above object, the present invention learns a scheduling neural network for each process of the factory workflow, simulates the factory workflow with a factory simulator, and uses the simulation results to learn the scheduling neural network for each process, including product type and A neural network-based process scheduling system independent of the number of types, comprising: a simulation execution unit that simulates the factory workflow using the factory simulator; a candidate type selection unit that selects product type candidates (hereinafter referred to as candidate types) in a workflow state simulated by the factory simulator, and selects as many candidate types as the number of product types in the scheduling neural network; a product type mapping unit that maps the selected candidate type and the product type of the scheduling neural network; And, generate learning data of the scheduling neural network using the workflow state simulated by the factory simulator, select a candidate type through the candidate type selection unit, and select the candidate type selected through the product type mapping unit and the scheduling. It is characterized by including a learning execution unit that maps the product type of the neural network and generates learning data corresponding to the corresponding workflow state.

In addition, the present invention is a neural network-based process scheduling system that is independent of product type and number of types, wherein the candidate type selection unit uses variables of the workflow state (hereinafter referred to as state variables) related to each product type to determine the corresponding state of each product type. It is characterized by obtaining the variable value of the variable and selecting the product type according to the variable value.

In addition, the present invention is a neural network-based process scheduling system independent of product type and number of types, wherein the candidate type selection unit calculates the evaluation value V(d) of each product type d using the following formula.

[Formula 1]

However, w _k represents the weight for the kth state variable, and v _k (d) represents the variable value of product type d for the kth state variable.

In addition, the present invention is a neural network-based process scheduling system independent of product type and number of types, wherein the state variable is one or more of machine deadline, number of waiting lots, equipment need, waiting time, and possibility of continuous operation. do.

In addition, the present invention is a neural network-based process scheduling system that is independent of product type and number of types, wherein the product type mapping unit always maps the product type of the scheduling neural network identically according to the order of selection of the selected product type. do.

In addition, the present invention relates to a neural network-based process scheduling system that is independent of product type and number of types, where the product type mapping unit generates a smaller number when the number of product types in the workflow state is less than the number of product types in the scheduling neural network. It is characterized by creating as many virtual product types (hereinafter referred to as virtual types), but mapping the state by setting the state of the corresponding virtual type to a state where there is no corresponding product.

In addition, the present invention relates to a neural network-based process scheduling system that is independent of product type and number of types, where the product type mapping unit generates a smaller number when the number of product types in the workflow state is less than the number of product types in the scheduling neural network. It is characterized by creating as many virtual product types (hereinafter referred to as virtual types) or creating duplicate product types.

In addition, the present invention relates to a neural network-based process scheduling system that is independent of product type and number of types. The system simulates the factory simulator and makes decisions for each process using the scheduling neural network for each process, thereby scheduling the process. A production episode is generated for each process, and for the scheduling neural network of each process, a candidate type is selected through the candidate type selection unit, and the candidate type is mapped to the product type of the scheduling neural network through the product type mapping unit to generate state data. It is characterized by further comprising a scheduling generator that generates and obtains a result by applying the generated state data to the scheduling neural network of the corresponding process.

In addition, the present invention is a neural network-based process scheduling system that is independent of product type and number of types, and the scheduling neural network for each process is a neural network based on reinforcement learning.

As described above, according to the neural network-based process scheduling system independent of the product type and number of types according to the present invention, candidate types are defined in advance with a finite number of N, candidates as many as the defined number are selected and applied to the neural network. By sharing the neural network even if the product types being combined into N are different, the effect of being able to use the previously learned neural network as is without redesigning/relearning the neural network even if the combined product type changes is achieved.

Figure 1 is an example diagram showing the structure of a neural network according to the number of product types.
Figure 2 is an exemplary diagram showing a model of a factory workflow according to an embodiment of the present invention.
Figure 3 is a block diagram of the configuration of a process according to an embodiment of the present invention.
Figure 4 is a diagram illustrating the equipment configuration of a process according to an embodiment of the present invention.
Figure 5 is an example table showing job replacement times according to an embodiment of the present invention.
Figure 6 is a configuration diagram of the entire system for implementing the present invention.
Figure 7 is a basic operational structure diagram of reinforcement learning used in the present invention.
Figure 8 is an example diagram showing the process of extracting state data of a factory workflow and applying it to a neural network according to an embodiment of the present invention.
Figure 9 is a block diagram of the configuration of a factory simulator-based scheduling system according to an embodiment of the present invention.
10 is a table illustrating state variables according to an embodiment of the present invention.
Figure 11 is an example diagram showing the process of selecting and mapping a product type according to an embodiment of the present invention.

Hereinafter, specific details for implementing the present invention will be described with reference to the drawings.

In addition, in explaining the present invention, like parts are given the same reference numerals, and repeated description thereof is omitted.

First, the configuration of the factory workflow model used in the present invention will be described with reference to FIGS. 2 to 5.

As shown in Figure 2, the factory workflow consists of a series of multiple processes, and one process is connected to another process. Additionally, connected processes have a precedence relationship. When viewing a process as a single node, the entire factory workflow forms a directed graph. Below, for convenience of explanation, process is used interchangeably with process node.

In the example of Figure 2, the factory workflow consists of processes P0, P1, P2, ..., P4, starting with process P0 and ending with process P4. When process P0 is completed, the next process P1 starts, and when process P1 is completed, the next process P2 starts.

Meanwhile, one lot is a specific workpiece and is produced as a finished product through each process P0, P1, P2, ..., P4 of the factory workflow. For a specific workpiece (=lot), when process P0 is completed, the lot can be delivered and the next process P1 can be started. In other words, when a lot (LOT) that has been processed in process P0 is provided to process P1, process P1 takes over the lot and continues to perform additional work. When all series of processes in the factory flow are processed in this way, the product of the corresponding lot is produced.

Additionally, the factory workflow does not produce only one product type (or product type, product family), but multiple types of products can be processed and produced. Therefore, each lot has a different type depending on the product type. For example, if Lot 1 is of type 'Product A', then the final product of Lot 1 is Product A. Additionally, if Lot 2 is of type 'Product B', Product B is produced as the final product of Lot 2.

Additionally, each process can be run simultaneously. For example, when lot 8 (product B) is being processed in process P4, lot 1 (or product A) may be intermediately processed in process P0 at the same time, and lot 2 (product B) may be processed in process P1. .

Next, the operation types of the process are explained.

Additionally, one process can selectively perform multiple task types. This process performs one of the possible work types for the input lot. At this time, a lot (hereinafter referred to as input lot) is input into the process, and as the work of the process is performed, the processed lot (hereinafter referred to as completed lot) is output (calculated). In other words, the lot on which work has been completed becomes the input lot that can be worked on in the next process Pn+1.

In the example of Figure 3, process Pn has multiple operation types, such as operation type n-1, operation type n-2, ..., operation type n-M. Process Pn selects and performs one task among M tasks. At that time, one of multiple tasks is selected and performed depending on the environment or request. In particular, preferably, if the process has a scheduling neural network, the task (or task type) (to be performed next) is selected from the neural network of the process and scheduled.

As an example, process Pn (n=0,1,2,3…) refers to an operation that collectively refers to several operation types (e.g. “assembly”), each operation type depending on the type of input lot (e.g. “Assembling legs” if the input lot type is chair, “Assembling top” if the input lot type is table, “Assembling drawers” if the input lot type is chest of drawers, etc.)

Meanwhile, preferably, two operation types can be carried out simultaneously in one process (eg process Pn). In other words, if there are M facilities capable of processing one process, work can be performed on M different product types.

Next, the equipment for the process will be explained.

Additionally, there may be multiple pieces of equipment deployed in process Pn, and one specific “equipment” is designated for each input lot for each work type. These designable relationship definitions are shown in Figure 4. In other words, the work types and lot types that can be performed for each specific piece of equipment are different, and one or multiple work types can be performed for each piece of equipment.

In the example of Figure 4, process P0 has three operation types, and each operation type P0-1, P0-2, and P0-3 are operations for processing product A, product B, and product C, respectively. Additionally, there are two pieces of equipment, Equipment 1 and Equipment 2, for working on work type P0-1 (or Product A). Additionally, the equipment for working types P0-2 and P03 are Equipment 2 and Equipment 1, respectively.

There may be multiple lots within the process Pn, and the lot status can be divided into a lot currently being worked on and a lot waiting for work to begin (waiting lot). The waiting lot status indicates that the equipment is already processing another lot in the process, or the equipment that can process the lot is not ready for work.

Meanwhile, as described above, work on two or more product types may be performed simultaneously in one process. In the example of Figure 4, Product B and Product C are performed by Equipment 2 and Equipment 1, respectively, so Product B and Product C can be performed simultaneously.

Next, job change in the process will be explained.

In process Pn, when specific equipment has different operation types for the previous lot and the next lot, the process of performing preparatory work and changing equipment settings in order to process the operation type of the next lot from the operation (or operation type) of the previous lot. Perform. This is called job change. Additionally, it takes a certain amount of time to change jobs.

For example, job replacement may be oil replacement, equipment drive program replacement, tool replacement within the equipment, equipment preheating/cooling work, etc.

The time required to replace this task is shown in Figure 5. In the example of FIG. 5, if the equipment that was currently producing the lot of operation type P0-1 wants to produce the lot of operation type P0-3, a job replacement operation that takes 12 hours must be performed. That is, after a 12-hour work change, the lot of the next work type P0-3 can be processed.

When changing work, even if there is a lot that can be put into the equipment, it cannot be produced (processed). Therefore, if job changes occur frequently, it takes a lot of time to change jobs, so the equipment utilization rate decreases.

In process Pn, only operation types that produce specific lot types can be continued without switching operations. In this case, since other lot types awaiting work cannot proceed to the completed lot, if there are multiple lot types requested by the customer, the appropriate “job change point” must be determined and work change must be performed.

Additionally, job replacement is performed on a equipment-by-equipment basis. Therefore, while one piece of equipment is being swapped for a different product type, a job swap can be performed on another piece of equipment.

Next, the configuration of the entire system for implementing the present invention will be described with reference to FIG. 6.

As shown in Figure 6, the entire system for implementing the present invention includes a neural network agent 10 consisting of a neural network 11, a factory simulator 20 that simulates the workflow of the factory, and a neural network agent 10. It consists of a scheduling system 30 that generates scheduling for each process. Additionally, it may be configured to further include a database 40 that stores scheduling data, etc.

First, the neural network agent 10 is composed of at least one neural network 11 that outputs the next task (or task action) of a specific process when the state of the workflow (or lot state) is input.

In particular, one neural network 11 is configured to determine the next task (or task type) for one process. That is, preferably, one of a number of operations (or types of operations) that can be performed next in the process is selected. As an example, the output of the neural network 11 consists of nodes corresponding to task types, the output of each node outputs a probability value, and the task corresponding to the node with the largest probability value is selected as the next task.

In other words, the scheduling artificial neural network 11 of each process makes a decision to select the “next work lot,” and can choose a different work type at the expense of work change time, or select a lot of the same work type again. .

Additionally, preferably, the neural network 11 makes decisions about the next task for the facility. For example, if there are two machines MC1 and MC2 in process P2, before MC1 ends the current work lot, it requests the neural network to make a decision on the next lot, and MC2 does the same at different times (or it may be the same time). ), you can request a decision from the neural network (11) independently. In other words, a neural network does not make decisions on multiple facilities simultaneously.

Additionally, in order to determine the next task of the multiple processes, multiple neural networks 11 for each of the multiple processes may be configured. In the example of FIG. 2, if there are five processes, a total of five processes can be configured by configuring the neural network 11 corresponding to each process. However, a neural network is not constructed for simple processes that do not require selection, such as when there is only one operation to select within the process.

Neural networks and their optimization use typical reinforcement learning-based neural network methods such as DQN (Deep-Q Network) [Non-patent Document 1]. That is, the neural network 11 can be learned using reinforcement learning as shown in FIG. 7.

In addition, the neural network agent 10 determines the workflow state (S _t ), the task in that state (a _t ), the workflow state after being performed by the task (S _t+1 ), and the task in that state. By receiving the compensation (r _t ) for , the parameters of the neural network (11) of the process are optimized.

In addition, when the neural network 11 is optimized (learned), the neural network agent 10 applies the workflow state (S _t ) to the optimized neural network 11 to output the next task (a _t ).

Meanwhile, the workflow status (S _t ) represents the workflow status (or factory status) at time t. Preferably, the workflow state consists of the state of each process within the workflow and the factory state corresponding to the entire factory. For example, the workflow status (St) consists of information representing the status of each process or factory, such as the distribution within the factory of the lot corresponding to the product type for which the decision is to be made and the current work product type of the production facility for which the decision is to be made.

Additionally, preferably, the workflow state may include only the states of some processes within the workflow. At this time, only core processes, such as those that cause bottlenecks within the workflow, can be targeted, and only the states of those processes can be included. Additionally, the workflow state is set to elements that change during the workflow. In other words, components that do not change as the workflow progresses are not set to status.

The status of each process (or process status) consists of the input lot, the status of each process equipment, etc. In addition, the factory status indicates the status of the entire process, such as the product production target and achieved status.

Meanwhile, as above, the status is set to the overall workflow state, and the action is set to the task in the corresponding process. In other words, the process state includes the placement status and equipment status of lots within the entire workflow, but the action (or task) is limited to a specific process node. In factories, the Theory of Constraint (TOC) states that when optimally scheduling a specific process node that is the bottleneck of production capacity or requires decision-making, the problems of related process nodes before and after are not concerned. Non-patent Document 2] is assumed. This is like making major decisions at major management points such as traffic lights, intersections, and interchanges, but for this purpose, the traffic situation on all connected roads before and after must be reflected as the state.

Meanwhile, compensation (r _t ) represents the compensation to be optimized through decision making in the factory. For example, compensation consists of facility operation rate (= time worked by the facility/total time), delivery satisfaction rate (= production volume that complies with the target time to complete work for each product/production amount to complete all work), etc. In other words, the compensation value can be formed by weighting and summing these factors.

That is, the reward (r _t ) in each process state (or lot state) (S _t ) is calculated using a reinforcement learning method. The reward (r _t ) in each state (S _t ) is calculated from the final result (or final performance) of the corresponding production episode. In other words, the final result (or final performance) is the main KPI used in factory management, such as the operation efficiency of the production equipment (equipment) of the process or overall workflow, the turn-around time (TAT) of the workpiece, and the production target achievement rate. It is calculated by (Key Performance Index), etc.

In summary, transitions can be extracted by extracting the state (S _t ), task (a _p,t ), and reward (r _t ) in chronological order from the production episode. In other words, a transition consists of the current state (S _t ) and task (a _p,t ) to the next state (S _t+1 ) and reward (r _t ). This means that when a specific process task (a _p,t ) is performed in the current state (S _t ), it transitions to the next state (S _t+1 ) and obtains the value of reward (r _t ). Here, reward (r _t ) refers to the value of the current state (S _t ) when the task (a _p,t ) is performed.

Meanwhile, for the state data (or input data, input node) and work data (or output data, output node) of the neural network 11 of each process, the number of product types is N (N is a natural number of 2 or more), which is defined in the dictionary. is set in In other words, the status data represents the status of each process or factory, and the status includes the status of products with different product types. At this time, the number of product types in the neural network 11 of each process is preset to N. Therefore, the form or format of the status data related to the product type is fixed, and the form, such as the number of work data, is also fixed. The number of product types at this time is called the effective number.

Meanwhile, the number (or effective number) of product types in the neural network 11 of each process may be different. That is, the number (or effective number) of product types in the neural network of process P2 may be 2, and the number of product types in the neural network of process P4 may be 3.

State data and task data are used in the neural network of a reinforcement learning model, but in the neural network of a general model, the workflow state and next task correspond to input data (or input node) and output data (output node), respectively.

Next, the factory simulator 20 is a typical simulator that simulates factory workflow.

The factory workflow uses the same workflow model as shown in Figure 2 above. In other words, the factory workflow model in the simulation is modeled as a directed graph consisting of a number of nodes representing the process. Each process model in the simulation is modeled with the actual facility status on site.

That is, as shown in Figure 3, each process model includes the lot (LOT) input to the process, the lot (LOT) completed in the process, multiple work types, equipment for each work, and processing of work according to each work type. Equipment configuration and processing capabilities, such as speed and work changeover time to replace the previous work type with the new work type, are modeled as modeling variables.

The above factory simulator employs conventional simulation technology. Therefore, more detailed explanations are omitted.

Next, the scheduling system 30 performs a simulation using the factory simulator 20, extracts learning data from the simulation results, and uses the extracted learning data to create a neural network agent 10, that is, the neural network 11 of each process. ) is learned. Additionally, the scheduling system 30 generates scheduling data for actual products to be produced using the factory simulator 20 and the learned neural network agent 10.

That is, the scheduling system 30 simulates multiple production episodes with the factory simulator 20. A production episode refers to the entire process of producing a final product (or lot). At this time, each production episode has a different processing process.

For example, performing a simulation to produce 100 red ballpoint pens and 50 blue ballpoint pens once is one production episode. At this time, the detailed processes processed within the factory workflow may be different. If the detailed process is different, another production episode is created. For example, processing a lot of red ballpoint pens and processing a lot of blue ballpoint pens in process 2 under a certain state are different production episodes.

Additionally, the scheduling system 30 can generate learning data using the generated episode and train the neural networks 11 through the neural network agent 10. That is, the scheduling system 30 simulates (mocks) the simulator 10 according to the production episode, extracts transitions from the simulation results, and constructs learning data. At this time, multiple transitions are extracted from one episode. Preferably, simulation is performed according to multiple episodes and a large amount of transitions are extracted from them.

Then, the scheduling system 30 applies the extracted learning data (or transitions) to the neural network agent 10 to learn it.

At this time, as an example, learning data (or transitions) can be learned sequentially in chronological order. Preferably, transitions are randomly sampled from all transitions, and the neural network agent 10 is trained using the sampled transitions.

Additionally, when the neural network agent 10 configures multiple neural networks, the corresponding neural network is trained using learning data (or transition data) of the process corresponding to each neural network.

Additionally, the scheduling system 30 generates scheduling data by simulating the actual product to be produced using the factory simulator 20. At this time, when simulating in the factory simulator 20, the decision to select the next task in each process is made by the neural network 11 of the process.

Meanwhile, the scheduling system 30 selects a valid number of product type candidates (or candidate types) from the workflow state of the factory simulator 20, and provides status data (or a neural network for each process) by the selected candidate types. state data) is mapped to the state data of the neural network. Additionally, the scheduling system 30 determines the candidate type corresponding to (mapped to) the result of the job data (or output data) of the neural network or the product type as the determined type and reflects the determined type in the factory simulator 20.

At this time, preferably, the scheduling system 30 selects a product type candidate from the workflow state of the factory simulator 20 according to preset rules.

Next, the database 40 is composed of a learning data DB 41 that stores learning data for training the neural network agent 10, a scheduling DB 42 that stores scheduling data, etc. However, the configuration of the database 40 is only a preferred embodiment, and when developing a specific device, it may be configured in a different structure according to database construction theory, taking into account ease of access and search and efficiency.

Next, an example in which factory workflow status and neural network status data are mapped by the simulator 10 according to an embodiment of the present invention will be described with reference to FIG. 8.

Figure 8 illustrates a case where the number of product types simulated by the simulator 10 is equal to the number of types of the neural network. In other words, there are two types of products simulated by the simulator 10, Product A and Product B, and the number of types of neural network is two, Product 1 and Product 2. In this case, it is possible to map directly from the factory workflow state to the state data of the neural network without separately selecting candidates.

In FIG. 8, (a) shows the workflow state on the simulator 10 as a result of simulation in the simulator 10, and (b) shows a neural network that makes decisions about the next task of process P4.

FIG. 8(a) shows the factory state (or factory workflow status information) at one of the factory states (or each process state) for the factory workflow in FIG. 2. Status information in factory workflows will change over time. The status of the factory workflow at a specific point in time consists of lot location and status, equipment status, etc.

In particular, as shown in Figure 8(a), the overall factory status consists of lot type, lot number (lot identification information), lot status, equipment, etc. for each process. The lot type indicates which product (or product type) the lot is for. Additionally, the lot number is information for identification as lot identification information. Additionally, lot status indicates whether the lot is waiting or in progress. Additionally, equipment refers to the equipment used when working that lot.

In other words, the status of the factory workflow at a specific point in time can be gauged by the location and status of the lot. In particular, it can indicate how many lots exist for each lot type per process and what their status is (working, waiting). Among these workflow states, the state data required for the neural network of process P4 is shown in the middle table of FIG. 8. That is, in the example of FIG. 8, the status data of the candidate product consists of lot distribution, equipment status, etc. Depending on the actual scheduling problem, additional information for each lot type (e.g., processing time, remaining order amount, etc.) different from the example in FIG. 8 may be added to the status of the factory workflow.

In addition, the neural network of process P4 in (b) of FIG. 8 is a neural network that makes decisions to select the next workpiece in process P4. In other words, it is an artificial neural network that makes a decision to select the next work piece when the machine has finished its current work in machine 1 (MC1) of process P4. Meanwhile, in the preceding processes P0 to P3 of the workflow, separate decisions may not be necessary, or a neural network that can already perform decision making may exist.

In the example of FIG. 8, equipment MC1 of process P4 is currently working on LOT7, and three pieces of equipment are each working on process P4. There are two product lot types, A and B, produced in the factory workflow in Figure 8. To enable the artificial neural network to make optimal decisions, the factory workflow status for each lot type and the situation for each equipment in process P4 are analyzed by the artificial neural network. Connect to In Figure 8 (a), the status information of the factory workflow is divided into product A/B to generate lot distribution status and equipment status, and these status data are converted into the input node of the artificial neural network or the status information of products 1 and 2 of the neural network. Map. That is, product A of the factory workflow maps to product 1 of the neural network, and product B of the factory workflow maps to product 2 of the neural network.

At this time, the output node or work data of the artificial neural network is arranged with nodes corresponding to the number of lot types, and the lot corresponding to the node with a strong size of the calculated signal (number) is the product (or product type) selected by the artificial neural network. do. The product type of the factory workflow corresponding to (mapped to) the output node of the neural network is finally selected. For example, if product 1 is selected in the neural network, product A, which maps to product 1, is selected in process P4. Additionally, when product 2 is selected in the neural network, product B, which is mapped to product 2, is selected in process P4.

Next, the configuration of a neural network-based process scheduling system independent of product type and number of types according to an embodiment of the present invention will be described with reference to FIG. 9.

As shown in Figure 9, the neural network-based process scheduling system 30, which is independent of product type and number of types according to an embodiment of the present invention, includes a simulation execution unit 31 that collects simulated data by simulating the simulator 10. , a candidate type selection unit 32 that selects a product type candidate in the workflow state, a product type mapping unit 33 that maps the candidate type of the workflow and the product type of the neural network, and generates learning data from the simulation results. It consists of a learning unit 34 that learns a neural network for a specific process. Additionally, it may be configured to further include a scheduling generator 35 that generates scheduling data.

First, the simulation execution unit 31 runs the simulator 10 to collect simulation data. As a result of simulation with the simulator 10, status information of the factory workflow can be collected. The status information of factory workflow changes over time. The status of the factory workflow at a specific point in time consists of the location and status of the lot, etc.

In particular, the simulation execution unit 31 can run the simulator 10 to collect the factory status (or each process status) for the factory workflow as shown in FIG. 8.

Next, the candidate type selection unit 32 selects a product type candidate in the workflow state simulated by the simulator 10.

That is, the candidate type selection unit 32 selects a valid number of candidates from among the product types simulated in the simulator 10. At this time, the effective number represents the number of product types of the neural network. Meanwhile, the candidate type selection unit 32 selects a product type candidate for each neural network of each process. In other words, candidates for different product types may be selected for each neural network in each process.

Preferably, the variable value (numerical value of the variable) of each product type is obtained using the variable (or state variable) of the workflow state related to each product type, and the product type is selected according to the variable value.

In the first method, the corresponding product type is evaluated by assigning weights to the variables of the workflow state related to each product type, and a valid number of product types are selected as candidates in order of the highest evaluation value.

The evaluation value V(d) of product type d is given by the following equation.

[Equation 1]

Here, w _k represents the weight for k variable, and v _k (d) represents the numerical value (or value, variable value) of the product type for k variable.

As shown in the table of FIG. 10, state variables for evaluating each product type may consist of delivery deadline, number of waiting lots, equipment need, waiting time, possibility of continuous operation, etc. The table in Figure 10 is only an example, and various variables can be used to select candidates.

For example, if the number of product types in the corresponding neural network is three, the three product types with the highest evaluation values are selected.

Also, as a second method, candidates can be selected by selecting product types with the highest values among each state variable. For example, if the number of product types is N=3, candidate 1 can be selected as a product type with highest priority on delivery date, candidate 2 as a product type with highest priority on equipment needs, candidate 3 as a product type with highest priority on continuous operation, etc. there is. In other words, the product types with the highest values of each variable are selected.

Next, the product type mapping unit 33 maps the candidate type of the workflow and the product type of the neural network. In other words, the state of the candidate type of the workflow is mapped to the state data of the neural network, or the product type of the result data (or task data) of the neural network is mapped to the product type of the next task of the corresponding process in the workflow.

As shown in Figure 11, there are five product types in the workflow running on the simulator 10: A, B, C, D, and E. At this time, the neural network of one process has three product types. In other words, the number of product types in the corresponding neural network is three.

At this time, three product types are selected from among the product types in the workflow. In the example of Figure 11, products B, C, and E are elected. At this time, the number of elected product types and the number of product types of the neural network are the same.

The state data (state in the workflow) of the selected product type is mapped to the state data of the product type in the neural network. In the previous example, Product B is mapped to Product 1, Product C is mapped to Product 2, and Product E is mapped to Product 3. In other words, the state of the selected product (state in the workflow) is generated as state data input to each neural network.

In other words, the artificial neural network makes decisions based on the order of the elected candidates without the need to know the actual product code/name defined inside the simulator.

Additionally, in Figure 11, the results of the product type selected by the neural network are mapped to the product type in the workflow and returned. In the example of Figure 11, the result of the neural network is product 2, and product 2 is mapped to product C.

In other words, the product type (or product sequence number) selected (or decision-making) in the neural network is mapped back to the actual product type (or product type in the workflow). In other words, the product type determined by the neural network is converted to the product type in the workflow and transmitted to the simulator.

Meanwhile, when learning data is generated by a production episode, the type of the next task by the production episode is mapped to the product type of the neural network's task data (next task, output data).

By using the same mapping process as before, the artificial neural network can always maintain the same structure (number of input and output nodes) regardless of changes in factory conditions (changes in the number of product types, changes in products produced by process). In addition, since the structure of the artificial neural network does not change, even if the number of products in the factory simulator changes or a new product is produced, the artificial neural network does not need to be retrained, and scheduling can be performed using the existing artificial neural network that has already been trained. there is.

In addition, selecting N candidates from all product types is possible at very low cost (calculation speed, computational resources, etc.), and the neural network is operated under the assumption that the probability of an optimal solution among these N is very high. In other words, the benefit of not reconstructing/relearning the artificial neural network even if the number of product types changes is much greater than the disadvantage of not finding an optimal solution among N candidates “with very low probability.”

Meanwhile, preferably, the product type mapping unit 33 always maps the order (or selection order) of product types selected in the candidate type selection unit 32 to the product types of the neural network in the same manner.

For example, in the first method, product types are arranged in descending order of evaluation value and mapped to the product type of the neural network according to the arranged order. Assume that the product types with the highest evaluation values in the first step are in the order B, C, and E, and the product types with the highest evaluation values in the second step are in the order C, E, and B. In this case, the selected product types are the same as B, C, and E, but the mapping order may be different. That is, in the first step, product type B is mapped to product 1 of the neural network, while in the second step, product type B is mapped to product 3 of the neural network.

If mapping is performed while maintaining the selection order as above, the learning of the neural network can be reflected more uniformly even if the product types are learned differently.

In addition, when the number of product types in the workflow is less than the number of product types in the neural network, the product type mapping unit 33 generates a smaller number of default product types (or virtual types), but the status of the virtual type is Set the product to a state where it does not exist and map it.

For example, assume that the number of product types in the workflow is 2 and the number of product types in the neural network is 3. In this case, one product type in the workflow is created as a virtual type. And if the status data of the virtual type is the lot number or the wait number, both the lot number and the wait number are treated as 0 and mapped. Additionally, delivery deadlines, etc. are mapped by setting them to an infinite period (or the longest deadline). In other words, the mapping assumes that there is no virtual type of product or corresponding lot.

Additionally, as another embodiment, the product type mapping unit 33 may select product types in duplicate when the number of product types in the workflow is less than the number of product types in the neural network. For example, if a factory produces two products, A and B, but there are three types of artificial neural network/candidate selection logic, candidate 1 A, candidate 2 B, and candidate 3 A are generated. That is, product type A is duplicated twice. Even in these cases, the neural network can find a clear optimal solution. In other words, in a situation where N=3 among products producing products A, B, C, D, and E, and there are no orders for products B, C, D, and E, product A can be selected for all candidates 1, 2, and 3. It means there is. In this case, the artificial neural network finally selects A with high probability.

Next, the learning execution unit 34 generates learning data from the collected simulation results, that is, the workflow state, and applies the learning data to the neural network of the corresponding process to train it.

In particular, the learning execution unit 34 selects a candidate type from the simulation results or workflow state simulated by the simulator 10 and generates learning data (state data or work data, etc.) of the neural network according to the selected candidate type. .

For example, when learning data is generated through one production episode, state data or work data of the neural network at each stage are generated according to the production episode. At this time, at each stage, the candidate type is selected through the candidate type selection unit 32, and the candidate type is mapped to the product type of the neural network through the product type mapping unit 33 to generate state data. Additionally, the product type of the next task in the corresponding production episode is mapped to the product type of the neural network to generate task data for the neural network. In this way, the state data and work data of the neural network are generated to generate transition data for learning.

Meanwhile, the learning execution unit 34 selects a candidate type for each production episode or step of each production episode. Therefore, the product types of learning data learned from the neural network may vary depending on each production episode or each stage.

However, the operation method of reinforcement learning adopted in the present invention selects the next action or output expected to result in the highest reward value/score given the given input or state. In other words, the basic purpose of reinforcement learning is not to estimate the action (output) corresponding to the correct answer for all inputs. The goal of reinforcement learning is to select the most appropriate action from a given input state, and during learning, an appropriate output (Action) is selected even when given an input state that has never been experienced.

In particular, in the case of three product types, decisions in the relevant process can be made based on the lot distribution status of the entire factory and the equipment status in a specific process, regardless of what type of product it is. The number of lot statuses for each candidate product from the previous process of the relevant process can be infinite. Therefore, even if the product type is not in the state at the time of learning, decisions that can improve the reward value/score can be made in the process. In other words, it can operate normally even if the product types during learning are A, B, and C, and the product types during actual operation are D, E, and F.

Next, the scheduling generator 35 creates a production episode for scheduling by simulating the process through the simulator 10 and making decisions for each process using a neural network.

At this time, the scheduling generation unit 35 selects a candidate type for the neural network of each process through the candidate type selection unit 32, and maps the candidate type to the product type of the neural network through the product type mapping unit 33. Generate status data. Then, the generated state data is applied to the neural network of the process to obtain the result (or next work data).

Additionally, the scheduling generator 35 reversely maps the job type of the obtained next job data to obtain the actual product type and applies it to the simulator 10.

Above, the invention made by the present inventor has been described in detail based on examples, but the present invention is not limited to the examples and, of course, can be changed in various ways without departing from the gist of the invention.

10: Neural network agent 11: Neural network
20: Factory Simulator 30: Learning System
31: Mock execution department 32: Candidate type selection department
33: Product type mapping unit 34: Learning execution unit
35: Scheduling creation unit
40: Database 41: Learning DB
42: Scheduling DB

Claims (8)

Neural network-based process scheduling independent of product type and number of types, where the scheduling neural network for each process of the factory workflow is learned, and the factory workflow is simulated with a factory simulator and the simulation results are used to learn the scheduling neural network for each process. In the system,
a simulation execution unit that simulates the factory workflow using the factory simulator;
a candidate type selection unit that selects product type candidates (hereinafter referred to as candidate types) in a workflow state simulated by the factory simulator, and selects as many candidate types as the number of product types in the scheduling neural network;
a product type mapping unit that maps the selected candidate type and the product type of the scheduling neural network; and,
Learning data of the scheduling neural network is generated using the workflow state simulated by the factory simulator, and a candidate type is selected through the candidate type selection unit, and the candidate type selected through the product type mapping unit is combined with the scheduling neural network. A neural network-based process scheduling system independent of product type and number of types, comprising a learning execution unit that maps product types and generates learning data corresponding to the corresponding workflow state.
According to paragraph 1,
The candidate type selection unit uses the variables of the workflow state (hereinafter referred to as state variables) related to each product type to obtain the variable value of the corresponding variable for each product type and selects the product type according to the variable value. A neural network-based process scheduling system that is independent of type and type count.
According to paragraph 2,
A neural network-based process scheduling system independent of product type and number of types, characterized in that the candidate type selection unit calculates the evaluation value V(d) of each product type d according to the following formula.
[Formula 1]

However, w _k represents the weight for the kth state variable, and v _k (d) represents the variable value of product type d for the kth state variable.
According to paragraph 2,
A neural network-based process scheduling system independent of product type and number of types, wherein the state variable is one or more of device deadline, number of waiting lots, equipment need, waiting time, and possibility of continuous operation.
According to paragraph 1,
A neural network-based process scheduling system independent of product type and number of types, wherein the product type mapping unit always maps the product type of the scheduling neural network identically according to the order of selection of the selected product type.
According to paragraph 1,
If the number of product types in the workflow state is less than the number of product types in the scheduling neural network, the product type mapping unit generates a smaller number of virtual product types (hereinafter referred to as virtual types) or creates duplicate product types. A neural network-based process scheduling system independent of product type and number of types.
According to paragraph 1,
The system generates production episodes for scheduling by simulating through the factory simulator and performing decision-making for each process with the scheduling neural network of each process, and for the scheduling neural network of each process, the candidate type selection unit. A candidate type is selected through the product type mapping unit, and the candidate type is mapped to the product type of the scheduling neural network to generate state data, and the generated state data is applied to the scheduling neural network of the process to obtain a result. A neural network-based process scheduling system independent of product type and number of types, further comprising a scheduling generator.
According to any one of claims 1 to 7,
A neural network-based process scheduling system independent of product type and number of types, characterized in that the scheduling neural network for each process is a neural network based on reinforcement learning.

Images