KR20230165026A - A factory simulator-based scheduling neural network learning system with lot history reproduction function - Google Patents

Abstract

Scheduling neural network learning based on a factory simulator with a lot history reproduction function that records the lot history immediately preceding a specific process among the simulated data generated by the simulator, generates various cases from the recorded lot history, and learns the neural network for the process. It relates to a system, comprising: a simulation execution unit that simulates the factory workflow using a factory simulator and collects simulated data; A lot history recording unit that simulates processes (hereinafter referred to as first stages) prior to a specific process through the simulation execution unit and records lot history information of the specific process; an episode reproduction unit that generates a plurality of second stage production episodes using the lot history information and simulates processes (hereinafter referred to as second stages) after the specific process according to the produced production episodes through the simulation execution unit; and a learning execution unit that generates learning data from the simulation results of the second stage and trains the scheduling neural network of the corresponding process.
By using the system as described above, the immediately preceding lot history of a specific process is recorded among the simulated data by the simulator and used for neural network learning of the corresponding process, so that learning data can be generated by generating various case numbers directly from the immediately preceding lot history, Through this, the learning process can proceed quickly by not repeating the simulation process up to the previous process.

Description

A factory simulator-based scheduling neural network learning system with lot history reproduction function }

The present invention learns a process-specific neural network for scheduling each process in a factory workflow consisting of a series of processes, and has a lot history reproduction function that simulates the factory workflow using a simulator and learns the neural network with simulated data. This study is about a factory simulator-based scheduling neural network learning system.

In particular, the present invention records the lot history immediately preceding a specific process among the simulated data generated by a simulator, and generates the number of various cases from the recorded lot history to learn the neural network of the process, a factory equipped with a lot history reproduction function. This is about a simulator-based scheduling neural network learning system.

Generally, manufacturing process management refers to activities that manage a series of processes performed during the manufacturing process from raw materials or materials to completion of a product. In particular, the processes and work sequences required to manufacture each product are determined, and the materials and time required for each process are determined.

In particular, factories that produce products are equipped with equipment that handles each process task, arranged in the work space for that process. The equipment can be configured to be supplied with lots to process specific tasks. In addition, transfer devices such as conveyors are installed between equipment or work spaces, so that the processed lot is moved to the next process when a specific process is completed by the equipment. In other words, one lot is produced as a finished product through a series of processes.

Additionally, in order to perform a specific process, multiple pieces of equipment with similar/same functions may be installed and the same or similar process tasks may be divided and processed. Scheduling the process or each task in such a manufacturing line is a very important issue for factory efficiency. Conventionally, most scheduling was done in a rule-based format according to each condition, but the performance evaluation of the scheduling results was ambiguous because the evaluation scale was not clear.

Additionally, recently, technologies for scheduling work by introducing artificial intelligence techniques into the manufacturing process have been proposed [Patent Document 1]. The above prior art used a machine learning algorithm called a genetic algorithm among artificial intelligence technologies, but limited the work of machine tools to scheduling. In addition, a technology applying a neural network learning method to processes of multiple facilities is also proposed [Patent Document 2]. However, the above prior art is a technology that finds the optimal control method in a given situation based on past data, and has a clear limitation in that it does not work without data accumulated in the past.

In order to solve the above problems, the present applicant proposes a technology for simulating the process using a factory simulator and learning a neural network for each process using the simulated data [Patent Document 3]. However, the prior art requires performing numerous simulation tasks with a factory simulator to cover all cases for learning. In particular, with respect to the processes preceding the specific process that is the subject of learning (the process corresponding to the neural network of the subject of learning), there are a number of different choices from the perspective of the process. Therefore, the prior art has a problem in that the previous processes must be repeated and simulated as many times as these cases.

Korean Patent Publication No. 10-1984460 (announced on May 30, 2019) Korean Patent Publication No. 10-2035389 (announced on October 23, 2019) Korean Patent Publication No. 10-2338304 (announced on December 13, 2021)

V. Mnih et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, p. 529, 2015. The Goal: A Process of Ongoing Improvement, Eliyahu M. Goldratt 1984

The purpose of the present invention is to solve the problems described above, by recording the immediately preceding lot history of a specific process among the simulated data generated by the simulator, and generating the number of various cases from the recorded lot history to create a neural network for the process. It provides a factory simulator-based scheduling neural network learning system with a lot history reproduction function.

In order to achieve the above object, the present invention provides a lot history reproduction function that learns the scheduling neural network for each process of the factory workflow and learns the scheduling neural network for each process with the simulation results of a factory simulator that simulates the factory workflow. It relates to a factory simulator-based scheduling neural network learning system, comprising: a simulation execution unit that simulates the factory workflow using the factory simulator and collects simulated data; A lot history recording unit that simulates processes (hereinafter referred to as first stages) prior to a specific process through the simulation execution unit and records lot history information of the specific process; an episode reproduction unit that generates a plurality of second stage production episodes using the lot history information and simulates processes (hereinafter referred to as second stages) after the specific process according to the produced production episodes through the simulation execution unit; and a learning execution unit that generates learning data from the simulation results of the second stage and trains a scheduling neural network for the corresponding process.

In addition, the present invention is a factory simulator-based scheduling neural network learning system with a lot history reproduction function, wherein the scheduling neural network of each process is configured to receive the state of the factory workflow and output the type of work to be processed next in that state. It is characterized by being

In addition, the present invention is a factory simulator-based scheduling neural network learning system with a lot history reproduction function, wherein the lot history information includes the time when each lot arrives at the corresponding process and becomes available for input (hereinafter referred to as arrival time). It is characterized by

In addition, the present invention is a factory simulator-based scheduling neural network learning system with lot history reproduction function, wherein each process can perform tasks of multiple task types and switch tasks from one task type to another task type. In order to do this, it is characterized by requiring a predetermined work replacement time.

In addition, the present invention is a factory simulator-based scheduling neural network learning system with a lot history reproduction function, wherein the episode reproduction unit generates a production episode according to selecting the lot that has arrived or is about to arrive as the next task from the lot history information. It is characterized by

In addition, the present invention is a factory simulator-based scheduling neural network learning system with a lot history reproduction function, wherein the scheduling neural network for each process is a neural network based on reinforcement learning.

As described above, according to the factory simulator-based scheduling neural network learning system with a lot history reproduction function according to the present invention, the immediately previous lot history of a specific process is recorded among the simulated data by the simulator and used for neural network learning of the process, Learning data can be generated by generating various case numbers directly from the previous lot history, and this allows the learning process to proceed quickly without repeating the simulation process up to the previous process.

1 is an exemplary diagram showing a model of a factory workflow according to an embodiment of the present invention.
Figure 2 is a block diagram of the configuration of a process according to an embodiment of the present invention.
Figure 3 is a diagram illustrating the equipment configuration of a process according to an embodiment of the present invention.
Figure 4 is an example table showing job replacement times according to an embodiment of the present invention.
Figure 5 is a configuration diagram of the entire system for implementing the present invention.
Figure 6 is a basic operational structure diagram of reinforcement learning used in the present invention.
7 is a table illustrating the status of factory workflow according to an embodiment of the present invention.
Figure 8 is a block diagram of the configuration of a factory simulator-based scheduling system according to an embodiment of the present invention.
Figure 9 is an exemplary diagram showing the division of stages according to a specific process according to an embodiment of the present invention.
10 is a table illustrating lot history information according to an embodiment of the present invention.

Hereinafter, specific details for implementing the present invention will be described with reference to the drawings.

In addition, in explaining the present invention, like parts are given the same reference numerals, and repeated description thereof is omitted.

First, the configuration of the factory workflow model used in the present invention will be described with reference to FIGS. 1 to 5.

As shown in Figure 1, a factory workflow consists of a series of multiple processes, and one process is connected to another process. Additionally, connected processes have a precedence relationship. When viewing a process as a single node, the entire factory workflow forms a directed graph. Below, for convenience of explanation, process is used interchangeably with process node.

In the example of Figure 1, the factory workflow consists of processes P0, P1, P2, ..., P4, starting with process P0 and ending with process P4. When process P0 is completed, the next process P1 starts, and when process P1 is completed, the next process P2 starts.

Meanwhile, one lot is a specific workpiece and is produced as a finished product through each process P0, P1, P2, ..., P4 of the factory workflow. For a specific workpiece (=lot), when process P0 is completed, the lot can be delivered and the next process P1 can be started. In other words, when a lot (LOT) that has been processed in process P0 is provided to process P1, process P1 takes over the lot and continues to perform additional work. When all series of processes in the factory flow are processed in this way, the product of the corresponding lot is produced.

Additionally, the factory workflow does not produce only one product type (or product type, product family), but multiple types of products can be processed and produced. Therefore, each lot has a different type depending on the product type. For example, if Lot 1 is of type 'Product A', then the final product of Lot 1 is Product A. Additionally, if Lot 2 is of type 'Product B', Product B is produced as the final product of Lot 2.

Meanwhile, preferably, the factory workflow may be different for each production group.

Additionally, each process can be run simultaneously. For example, when lot 8 (product B) is being processed in process P4, lot 1 (or product A) may be intermediately processed in process P0 at the same time, and lot 2 (product B) may be processed in process P1. .

Next, the operation types of the process are explained.

Additionally, one process can selectively perform multiple task types. The process works on “one of the work types” that can be performed on the input lot. At this time, a lot (hereinafter referred to as input lot) is input into the process, and as the work of the process is performed, the processed lot (hereinafter referred to as completed lot) is output (calculated). In other words, the lot on which work has been completed becomes the input lot that can be worked on in the next process Pn+1.

In the example of Figure 2, process Pn has multiple operation types, such as operation type n-1, operation type n-2, ..., operation type n-M. Process Pn selects and performs one task among M tasks. At that time, one of multiple tasks is selected and performed depending on the environment or request. In particular, preferably, if the process has a scheduling neural network, 'one task' is selected from the neural network of the process and scheduled.

As an example, process Pn (n=0,1,2,3…) refers to an operation that collectively refers to several operation types (e.g. “assembly”), each operation type depending on the type of input lot (e.g. “Assembling legs” if the input lot type is chair, “Assembling top” if the input lot type is table, “Assembling drawers” if the input lot type is chest of drawers, etc.)

Next, the equipment for the process will be explained.

Additionally, there may be multiple pieces of equipment deployed in process Pn, and one specific “equipment” is designated for each input lot for each work type. These designable relationship definitions are shown in Figure 3. In other words, the work types and lot types that can be performed for each specific piece of equipment are different, and one or multiple work types can be performed for each piece of equipment.

In the example of Figure 3, process P0 has three operation types, and each operation type P0-1, P0-2, and P0-3 are operations for processing product A, product B, and product C, respectively. Additionally, there are two pieces of equipment, Equipment 1 and Equipment 2, for working on work type P0-1 (or Product A). Additionally, the equipment for working types P0-2 and P03 are Equipment 2 and Equipment 1, respectively.

There may be multiple lots within the process Pn, and the lot status can be divided into a lot currently being worked on and a lot waiting for work to begin (waiting lot). The waiting lot status indicates that the equipment is already processing another lot in the process, or the equipment that can process the lot is not ready for work.

Next, job change in the process will be explained.

In process Pn, when specific equipment has different operation types for the previous lot and the next lot, the process of performing preparatory work and changing equipment settings in order to process the operation type of the next lot from the operation (or operation type) of the previous lot. Perform. This is called job change. Additionally, it takes a certain amount of time to change jobs.

For example, job replacement may be oil replacement, equipment drive program replacement, tool replacement within the equipment, equipment preheating/cooling work, etc.

The time required to replace this task is shown in Figure 4. In the example of FIG. 4, if the equipment that is currently producing a lot of job type P0-1 wants to produce a lot of job type P0-3, a job replacement operation that takes 12 hours must be performed. That is, after a 12-hour work change, the lot of the next work type P0-3 can be processed.

When changing work, even if there is a lot that can be put into the equipment, it cannot be produced (processed). Therefore, if job changes occur frequently, it takes a lot of time to change jobs, so the equipment utilization rate decreases.

In process Pn, only operation types that produce specific lot types can be continued without switching operations. In this case, since other lot types awaiting work cannot proceed to the completed lot, if there are multiple lot types requested by the customer, the appropriate “job change point” must be determined and work change must be performed.

Next, the configuration of the entire system for implementing the present invention will be described with reference to FIG. 5.

As shown in Figure 5, the entire system for implementing the present invention includes a neural network agent 10 consisting of a neural network 11, a factory simulator 20 that simulates the workflow of the factory, and a neural network agent 10. It consists of a learning system 30 for learning. Additionally, it may be configured to further include a database 40 that stores learning data, etc.

First, the neural network agent 10 is composed of at least one neural network 11 that outputs the next task (or task action) of a specific process when the state of the workflow (or lot state) is input.

In particular, one neural network 11 is configured to determine the next task (or task type) for one process. That is, preferably, one of a number of operations (or types of operations) that can be performed next in the process is selected. As an example, the output of the neural network 11 consists of nodes corresponding to all task types, the output of each node outputs a probability value, and the task corresponding to the node with the largest probability value is selected as the next task.

In other words, the scheduling artificial neural network 11 of each process makes a decision to select the “next work lot,” and can choose a different work type at the expense of work change time, or select a lot of the same work type again. .

Additionally, in order to determine the next task of the multiple processes, multiple neural networks 11 may be configured for each of the multiple processes. In the example of FIG. 1, if there are five processes, a total of five processes can be configured by configuring the neural network 11 corresponding to each process. However, a neural network is not constructed for simple processes that do not require selection, such as when there is only one operation to select within the process.

Neural networks and their optimization use typical reinforcement learning-based neural network methods such as DQN (Deep-Q Network) [Non-patent Document 1]

In addition, the neural network agent 10 determines the workflow state (S _t ), the task in that state (a _t ), the workflow state after being performed by the task (S _t+1 ), and the task in that state. By receiving the compensation (r _t ) for , the parameters of the neural network (11) of the process are optimized.

In addition, when the neural network 11 is optimized (learned), the neural network agent 10 applies the workflow state (S _t ) to the optimized neural network 11 to output the next task (a _t ).

Meanwhile, the workflow status (S _t ) represents the workflow status (or factory status) at time t. Preferably, the workflow state consists of the state of each process within the workflow and the factory state corresponding to the entire factory. For example, the workflow status (S _t ) consists of information representing the status of each process or factory, such as distribution within the factory of lots corresponding to the product type for which the decision is to be made and the current work product type of the production facility for which the decision is to be made. .

Additionally, preferably, the workflow state may include only the states of some processes within the workflow. At this time, only core processes, such as those that cause bottlenecks within the workflow, can be targeted, and only the states of those processes can be included. Additionally, the workflow state is set to elements that change during the workflow. In other words, components that do not change as the workflow progresses are not set to status.

The status of each process (or process status) consists of the input lot, the status of each process equipment, etc. In addition, the process status indicates the status of the entire process, such as the product production target and achieved status.

Meanwhile, as above, the status is set to the overall workflow state, and the action is set to the task in the corresponding process. In other words, the process state includes the placement status and equipment status of lots within the entire workflow, but the action (or task) is limited to a specific process node. A task or action a _p,t is an action or task at time t in process p, and represents the task of making a decision, i.e. the type of product selected, etc.

In other words, the process state includes the placement status and equipment status of lots within the entire workflow, but the action (or task) is limited to a specific process node. In factories, the Theory of Constraint (TOC) states that when optimally scheduling a specific process node that is the bottleneck of production capacity or requires decision-making, the problems of related process nodes before and after are not concerned. Non-patent Document 2] is assumed. This is like making major decisions at major management points such as traffic lights, intersections, and interchanges, but for this purpose, the traffic situation on all connected roads before and after must be reflected as the state.

Meanwhile, compensation (r _t ) represents the compensation to be optimized through decision making in the factory. For example, compensation consists of facility operation rate (= time worked by the facility/total time), delivery satisfaction rate (= production volume that complies with the target time to complete work for each product/production amount to complete all work), etc. In other words, the compensation value can be formed by weighting and summing these factors.

Next, the factory simulator 20 is a typical simulator that simulates factory workflow.

The factory workflow uses the same workflow model as shown in Figure 1 above. In other words, the factory workflow model in the simulation is modeled as a directed graph consisting of a number of nodes representing the process. However, each process model in the simulation is modeled with the actual facility status on site.

That is, as shown in Figure 2, the process model includes the lot (LOT) input to the process, the lot (LOT) completed in the process, multiple work types, equipment for each work type, and processing of work according to each work type. Equipment configuration and processing capabilities, such as speed and work changeover time to replace the previous work type with the new work type, are modeled as modeling variables.

The above factory simulator employs conventional simulation technology. Therefore, more detailed explanations are omitted.

Next, the learning system 30 performs a simulation using the factory simulator 20, extracts learning data from the simulation results, and uses the extracted learning data to create a neural network agent 10, that is, the neural network 11 for each process. ) is learned.

That is, the learning system 30 simulates multiple production episodes with the factory simulator 20. A production episode refers to the entire process of producing a final product (or lot). At this time, each production episode has a different processing process.

For example, performing a simulation to produce 100 red ballpoint pens and 50 blue ballpoint pens once is one production episode. At this time, the detailed processes processed within the factory workflow may be different. If the detailed process is different, another production episode is created. For example, processing a lot of red ballpoint pens and processing a lot of blue ballpoint pens in process 2 under a certain state are different production episodes.

Preferably, the learning system 30 can learn using reinforcement learning as shown in FIG. 6 [Non-patent Document 1]. That is, the reward (r _t ) in each process state (or lot state) (S _t ) is calculated using a reinforcement learning method. The reward (r _t ) in each state (S _t ) is calculated from the final result (or final performance) of the corresponding production episode. In other words, the final result (or final performance) is the main KPI used in factory management, such as the operation efficiency of the production equipment (equipment) of the process or overall workflow, the turn-around time (TAT) of the workpiece, and the production target achievement rate. It is calculated by (Key Performance Index), etc.

Additionally, transitions can be extracted by extracting the state (S _t ), task (a _{p, t} ), and reward (r _t ) according to time order from the production episode. In other words, a transition consists of the current state (S _t ) and task (a _p,t ) to the next state (S _t+1 ) and reward (r _t ). This means that when a specific process task (a _p,t ) is performed in the current state (S _t ), it transitions to the next state (S _t+1 ) and obtains the value of reward (r _t ). Here, reward (r _t ) refers to the value of the current state (S _t ) when the task (a _p,t ) is performed.

As above, the learning system 30 simulates (mocks) the simulator 10 according to the production episode, extracts transitions from the simulation results, and constructs learning data. At this time, multiple transitions are extracted from one episode. Preferably, simulation is performed according to multiple episodes and a large amount of transitions are extracted from them.

In addition, the learning system 30 records the history of the lot state immediately before a specific process in the simulated data simulated by the simulator 10, can generate a number of production episodes from the recorded lot state history data, and produces the production episodes. Simulate according to and extract learning data (or transition) based on the simulation results.

Then, the learning system 30 applies the extracted learning data (or transitions) to the neural network agent 10 to learn it.

At this time, as an example, learning data (or transitions) can be learned sequentially in chronological order. Preferably, transitions are randomly sampled from all transitions, and the neural network agent 10 is trained using the sampled transitions.

Additionally, when the neural network agent 10 configures multiple neural networks, the corresponding neural network is trained using learning data (or transition data) of the process corresponding to each neural network.

Next, the database 40 is composed of a learning data DB 41 that stores learning data for training the neural network agent 10, a lot history DB 42 that stores lot history, etc. However, the configuration of the database 40 is only a preferred embodiment, and when developing a specific device, it may be configured in a different structure according to database construction theory, taking into account ease of access and search and efficiency.

Preferably, the training data consists of multiple transitions. In particular, transition data can be classified by process. As before, the learning system 30 can collect a large amount of diverse transition data by simulating multiple episodes using the simulator 20 or by creating and simulating multiple episodes from lot history.

Next, an example in which a factory workflow is simulated by the simulator 10 according to an embodiment of the present invention will be described with reference to FIG. 7.

Figure 7 is a result of simulation with the simulator 10 and shows status information of the factory workflow. Status information in factory workflows will change over time. The state of the factory workflow at a specific point in time consists of the location and status of the lot.

In particular, FIG. 7 shows the factory state at one point among the factory states (or each process state) for the factory workflow of FIG. 1. As shown in Figure 7, the overall factory status consists of lot type for each process, lot number (lot identification information), lot status, work progress time, etc. The lot type indicates which product (or product type) the lot is for. Additionally, the lot number is information for identification as lot identification information. Additionally, lot status indicates whether the lot is waiting or in progress. In addition, the task progress time is the progress time after starting the task, and if the task is in progress, it indicates how much time it has taken since the task was started.

In the above state, the process of generating production episodes for learning the neural network of process P4 will be described. In other words, the neural network of process P4 is a neural network that makes decisions to select the next workpiece in process P4. In the preceding processes P0 to P3 of the workflow, separate decisions are not required, or an artificial neural network that can already make decisions exists.

Process P4 is currently working on lot 9 (product B), and P4 can generate various production episodes after lot 9 depending on the selection of the next workpiece. In particular, preferably, the following three production episodes can be generated.

[Production Episode 1] Lot 7

[Production Episode 2] Lot 8

[Production Episode 3] Lot 6

That is, production episodes 1 and 2 are episodes that select immediately available lot 7 and lot 8, respectively. Additionally, production episode 3 is an episode in which no work is performed on the equipment of process P4 and the production waits in preparation for the arrival of lot 6.

In the case of production episodes 1 and 2, that is, when process P4 selects LOT 7 or LOT 8, the job type changes, causing a JOB CHANGE. In other words, in process P4, product B is changed to product A or product C, so a replacement of production equipment (JOB CHANGE) occurs.

In the case of Production Episode 3, it is a case of waiting some time without selecting LOT 7 or LOT 8. In this case, lot 6 has product type B, so no JOB CHANGE occurs. In other words, the episode can sequentially process the same products, LOT 6, LOT 4, and LOT 2.

Therefore, production episode 3 will produce more efficient and productive results than production episodes 1 and 2. In other words, job replacement occurs at the same time that LOT 7 is selected as the next work in process P4. Therefore, if the arrival time of process P4 of LOT 6 is calculated as a result of the simulation for process P3, it is possible to determine through learning whether it was the right decision to select LOT7 or whether it was the right decision to select LOT6 after waiting for the equipment.

However, from the perspective of process P4, it is not possible to know in advance when LOT 6, LOT 4, and LOT 2 enter process P4 (when they become available for process P4). In other words, the completion time for each LOT existing in processes P0~P3 and the workable time for process P4 can only be known after completing all simulations for processes P0~P3.

Additionally, in this learning process, there are a number of cases of various production episodes from the perspective of process P4, and in order to simulate all the production episodes in this case, multiple “simulation of processes P0 to P4 of the factory workflow” is required, and excessive time is required. It takes. In the previous example, in order to simulate all three production episodes, the same process of processes P0 to P3 must be simulated repeatedly three times. If the process is complex, some of the same processes must be simulated over and over again for a very large number of episodes.

Next, the configuration of a factory simulator-based scheduling neural network learning system with a lot history reproduction function according to an embodiment of the present invention will be described with reference to FIG. 8.

As shown in FIG. 8, the factory simulator-based scheduling neural network learning system 30 with a lot history reproduction function according to an embodiment of the present invention includes a simulation execution unit 31 that collects simulated data by simulating the simulator 10. ), a lot history recorder 32 that records the history of the lot status of a specific process, an episode reproduction unit 33 that generates a number of episodes for a specific process from the lot history and simulates them according to the generated episodes, and simulation results. It consists of a learning unit 34 that generates learning data from and trains a neural network for a specific process.

First, the simulation execution unit 31 runs the simulator 10 to collect simulation data. As a result of simulation with the simulator 10, status information of the factory workflow can be collected. Status information in factory workflows will change over time. The state of the factory workflow at a specific point in time consists of the location and status of the lot.

In particular, the simulation execution unit 31 can run the simulator 10 to collect the factory status (or each process status) for the factory workflow as shown in FIG. 7.

Next, the lot history recording unit 32 simulates the processes up to the specific process through the simulation execution unit 31 and records the history of the lot status of the specific process.

As shown in FIG. 9, processes prior to a specific process will be referred to as the first stage, and processes following the specific process will be referred to as the second stage. In the example of FIG. 9, the first stage consists of processes P0, P1, P2, and P3, and the second stage consists of processes P4. If a process such as P5 exists after process P4, the second stage will include process P5, etc.

That is, the lot history recording unit 32 simulates the processes of the first stage (processes prior to a specific process) through the simulation execution unit 31 and records the history of the input lot of the specific process. do.

History information for the input lot of a specific process is illustrated in FIG. 10.

As shown in Figure 10, the history information of the input lot of a specific process consists of the time when the lots of each process in the first stage arrive at the process. In other words, lot history information consists of lot identification information (lot number), lot type of the corresponding lot, arrival time of the corresponding process (possible input time), etc.

Additionally, preferably, the lot history recording unit 32 also stores simulation results for the first stage.

Next, the episode reproduction unit 33 generates a production episode for the second stage (processes after a specific process) from the lot history information and performs a simulation according to the production episode.

Specifically, the episode reproduction unit 33 simulates the arrival of the input lot at the second stage (or a specific process) using lot history information, generates various production episodes according to the arrived input lot, and generates various production episodes according to the production episode. The processes of the second stage (processes following a specific process) are simulated through the simulation execution unit 31.

That is, the episode reproduction unit 33 reproduces the arrival time of each lot in the lot history information and simulates the arrival of the input lot in a specific process. In addition, the episode reproduction unit 33 generates a plurality of production episodes according to the arrival of the input lot, and simulates processes after a specific process according to the production episode through the simulation execution unit 31. Additionally, the episode reproduction unit 33 collects the simulation results of the second stage.

When running a simulation for neural network learning, the episode reproduction unit 33 generates lots by arrival time for the second stage, and plays the role of reproducing the arrival time of the lot as a simulation result of the first stage.

When learning the artificial neural network for decision-making in process P4, it is unnecessary to perform additional simulations for the first stage. Lots are created at the necessary time according to the lot history record as shown in Figure 10, and only the simulation for the second stage is repeated. do.

In the artificial neural network learning stage, which requires hundreds to tens of thousands of simulations, there is no need to simulate the entire first stage, so the simulation execution time can be shortened and the completion time of neural network learning can be shortened. In other words, the time required to learn the artificial neural network to be installed in the P4 process, which is the subsequent process of the workflow, can be significantly shortened.

Next, the learning execution unit 34 generates learning data from the collected simulation results and applies it to the neural network of the corresponding process to learn it.

In particular, the learning execution unit 34 generates a production episode by combining the production episode of the first stage with the production episode of the second stage, and applies the simulation result according to the generated final production episode to the neural network of the corresponding process. In the previous example, the first stage has one production episode, and the second stage has three episodes. One first stage production episode is combined with three second stage episodes, so a total of three final production episodes are produced. And the simulation results of the first and second stages are also the simulation results of the final production episode.

Above, the invention made by the present inventor has been described in detail based on examples, but the present invention is not limited to the examples and, of course, can be changed in various ways without departing from the gist of the invention.

10: Neural network agent 11: Neural network
20: Factory Simulator 30: Learning System
31: Mock execution unit 32: Lot history record unit
33: Episode reproduction unit 34: Learning execution unit
40: Database 41: Learning data DB
42: Lot history DB

Claims (6)

A factory simulator-based scheduling neural network learning system with a lot history reproduction function that learns the scheduling neural network of each process of the factory workflow and learns the scheduling neural network of each process using the simulation results of a factory simulator that simulates the factory workflow. In
a simulation execution unit that simulates the factory workflow using the factory simulator and collects simulated data;
A lot history recording unit that simulates processes (hereinafter referred to as first stages) prior to a specific process through the simulation execution unit and records lot history information of the specific process;
an episode reproduction unit that generates a plurality of second stage production episodes using the lot history information and simulates processes (hereinafter referred to as second stages) after the specific process according to the produced production episodes through the simulation execution unit; and,
A factory simulator-based scheduling neural network learning system with a lot history reproduction function, comprising a learning execution unit that generates learning data from the simulation results of the second stage and trains the scheduling neural network of the corresponding process.
According to paragraph 1,
A scheduling neural network learning system based on a factory simulator with a lot history reproduction function, wherein the scheduling neural network for each process is configured to output the type of work to be processed next in that state when the state of the factory workflow is input.
According to paragraph 2,
A factory simulator-based scheduling neural network learning system with a lot history reproduction function, wherein the lot history information includes the time when each lot arrives at the relevant process and becomes ready for input (hereinafter referred to as arrival time).
According to paragraph 3,
Each of the above processes can perform multiple work types, and is equipped with a lot history reproduction function, which requires a predetermined work change time to switch from one work type to another work type. A scheduling neural network learning system based on a factory simulator.
According to clause 4,
A factory simulator-based scheduling neural network learning system with a lot history reproduction function, wherein the episode reproduction unit generates a production episode by selecting a lot that has arrived or is about to arrive as a next task from the lot history information.
According to any one of claims 1 to 5,
A factory simulator-based scheduling neural network learning system with a lot history reproduction function, wherein the scheduling neural network for each process is a neural network based on reinforcement learning.

Images