CN112766530A - Production preparation and production start operation auxiliary device, system and method - Google Patents
Production preparation and production start operation auxiliary device, system and method Download PDFInfo
- Publication number
- CN112766530A CN112766530A CN201911075243.0A CN201911075243A CN112766530A CN 112766530 A CN112766530 A CN 112766530A CN 201911075243 A CN201911075243 A CN 201911075243A CN 112766530 A CN112766530 A CN 112766530A
- Authority
- CN
- China
- Prior art keywords
- production
- action
- unit
- production start
- preparation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 440
- 238000000034 method Methods 0.000 title claims abstract description 96
- 238000002360 preparation method Methods 0.000 title claims abstract description 96
- 230000009471 action Effects 0.000 claims abstract description 209
- 230000006870 function Effects 0.000 claims abstract description 52
- 230000002787 reinforcement Effects 0.000 claims abstract description 13
- 238000007726 management method Methods 0.000 claims description 34
- 238000012384 transportation and delivery Methods 0.000 claims description 28
- 238000005457 optimization Methods 0.000 claims description 22
- 239000000463 material Substances 0.000 claims description 14
- 238000007689 inspection Methods 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 8
- 125000004122 cyclic group Chemical group 0.000 claims description 6
- 239000002994 raw material Substances 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 claims description 5
- 238000012423 maintenance Methods 0.000 claims description 5
- 238000004140 cleaning Methods 0.000 claims description 3
- 239000013071 indirect material Substances 0.000 claims description 3
- 230000008569 process Effects 0.000 description 40
- 238000004364 calculation method Methods 0.000 description 34
- 238000003860 storage Methods 0.000 description 19
- 238000010586 diagram Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 230000008859 change Effects 0.000 description 7
- 230000003247 decreasing effect Effects 0.000 description 7
- 230000006872 improvement Effects 0.000 description 7
- 239000003795 chemical substances by application Substances 0.000 description 4
- 230000002950 deficient Effects 0.000 description 4
- 238000003326 Quality management system Methods 0.000 description 3
- 238000011960 computer-aided design Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000004904 shortening Methods 0.000 description 3
- 238000010924 continuous production Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000013439 planning Methods 0.000 description 2
- 238000000275 quality assurance Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 239000010730 cutting oil Substances 0.000 description 1
- 230000001351 cycling effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06316—Sequencing of tasks or work
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0633—Workflow analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/105—Human resources
- G06Q10/1057—Benefits or employee welfare, e.g. insurance, holiday or retirement packages
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Development Economics (AREA)
- Quality & Reliability (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Data Mining & Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- General Factory Administration (AREA)
Abstract
The invention provides a production preparation and production start operation auxiliary device, system and method, which is used for assisting production preparation and production start operation of a production line with a plurality of production devices, and comprises the following steps: a state information acquiring unit for acquiring state information including at least 4M from the production line; an action judgment part which determines and makes the production line execute the next production starting action instruction by using the action strategy algorithm of reinforcement learning based on the action value function and according to the state information; a judgment information acquisition unit for acquiring judgment information including at least QCD from the production line after the next production start action instruction is executed; and a learning unit that compares the judgment information with each of the threshold values of the judgment information, updates the action merit function based on a reward calculated based on the comparison result and the state information acquired after execution of the next production start action instruction, and provides the updated action merit function to the action judgment unit.
Description
Technical Field
The present invention relates to a production preparation and production start job assisting device, system and method, and more particularly to a production preparation and production start job assisting device, a production preparation and production start job assisting system and a production preparation job assisting method, which can improve a production QCD based on information such as production materials, production facilities, job methods and workers on a production line even during variable production in manufacturing industry.
Background
In the manufacturing industry, as shown in the production step of fig. 7, when a plan is prepared for a production preparation job and a production start job indicated by a dashed line frame, a preparation job of each device used in each process takes a lot of time and cost. For this reason, the following proposals have been made: a plan is created for a production preparation job by a skilled worker using experience (for example, a know-how) of the skilled worker according to a situation specific to each plant of the respective plants, and the plan is implemented.
In terms of process improvement, it is necessary to plan and implement a process for improving QCD (Quality, Cost, Delivery date) of a product, or a process for shortening preparation Time and C/T (Cycle Time) before production of a production facility, for example, for changing and reorganizing the arrangement of each production facility on a production line.
In addition, in terms of technical improvement, it is necessary to introduce a technique for improving bottleneck techniques of products and facilities, or to calculate an optimum value of 4M (Material, Machine, Method, Man: production Material, production facility, operation Method, operator) conditions of a facility line. In the 4M condition proposing work for calculating the optimum operating conditions of 4M, it is necessary to set 4M conditions to be approximate references based on experience and perform trial production work, and it is necessary for a production technician to adjust various 4M conditions to the optimum 4M conditions while checking the state of the product with reference to QCD data, thereby constructing a production line for final mass production.
For example, as shown in patent document 1, as a technique for assisting a production technician in setting a 4M condition, there are proposed a technique for storing and comparing 4M conditions and QCD data, a technique for reading and using past 4M conditions in response to a request from an operator, and the like.
Documents of the prior art
Patent document
Patent document 1: japanese patent laid-open publication No. 2018-97569
Disclosure of Invention
Technical problem to be solved by the invention
However, when devising and managing what kind of improvement measures are to be taken in each step, that is, when improving the production QCD in the production line, it takes a lot of time to flexibly operate the know-how of an experienced technician such as a skilled worker.
In particular, in the process of variable production, particularly, mass production of a plurality of types and small quantities, it is necessary to adjust the production line and the production process for each type of product to be produced, and it is required to further shorten the production preparation time of the QCD, for example, the time for production preparation switching, etc. for the user.
In addition, in a job in which an operation condition is proposed by an operator, a difference in technical ability of the operator who performs the job may cause a large amount of time to be taken until the optimal operation condition is calculated, or cause a difference in level (quality) of the optimal operation condition proposed by the operator. Therefore, when a production technician performs a start-up operation (QCD improvement operation) of a production line, a lot of time is consumed until the optimum QCD is reached or a difference in the level of optimum start-up (QCD) is generated by the production technician due to a difference in technical ability and experience of the production technician performing the operation, and thus, there are problems in that: it is difficult to start up with the same QCD for each production line.
In addition, in the operation condition proposing work, it is also important to derive a 4M state that can suppress the cost from the viewpoint of the manufacturing cost of the product at the time of mass production. However, there are the following problems: even for a skilled production technician, it is difficult to derive 4M conditions for performing production in a manner that can keep the quality of a product high and suppress costs.
The present invention has been made in view of the above problems, and an object of the present invention is to provide a production preparation and production start-up work assisting device, a production preparation and production start-up work assisting system including the same, and a production preparation and production start-up work assisting method, which improve a production QCD and shorten a production preparation time by optimizing a flow from the first planning of a production preparation work to the final execution of a production start-up work by machine learning.
Technical scheme for solving technical problem
In order to solve the above-described problems, a production preparation and production start operation support device according to a first aspect of the present invention supports production preparation and production start operation of a production line having a plurality of production facilities, the device including: a state information acquisition unit that acquires state information including at least a production material, a production facility, a work method, and a worker from a production line; an action judging part which determines the next production starting action instruction by using the action strategy algorithm of reinforcement learning according to the state information from the state information acquiring part based on the action value function and makes the production line execute the next production starting action instruction; a judgment information acquisition unit that acquires judgment information including at least quality, cost, and delivery date from the production line after the production line executes a next production start action instruction; and a learning unit that compares the judgment information from the judgment information acquisition unit with the threshold value of each judgment information, calculates a reward based on the comparison result, updates the action merit function based on the calculated reward and the status information acquired by the status information acquisition unit after execution of the next production start action instruction, and provides the updated action merit function to the action judgment unit.
In order to solve the above problem, a production preparation and production start-up work support system according to a second aspect of the present invention includes: a production preparation and production start-up work assisting apparatus according to a first aspect of the present invention; and a communication unit that transmits the status information, the judgment information, the next production start action instruction, and the action cost function to an external device connected to the production preparation and production start operation support system via a network.
In order to solve the above-described problems, a production preparation and production start-up work assisting method according to a third aspect of the present invention is a method for assisting production preparation and production start-up work in a production line having a plurality of production facilities, the method including: a state information acquisition step of acquiring state information including at least a production material, production equipment, an operation method, and an operator from a production line; an action judgment step of determining a next production start action instruction by using an action strategy algorithm of reinforcement learning according to the state information acquired in the state information acquisition step based on an action value function, and causing the production line to execute the next production start action instruction; a judgment information acquisition step of acquiring judgment information including at least quality, cost, and delivery date from the production line after the production line has executed a next production start action instruction; and a learning step of comparing the judgment information acquired in the judgment information acquisition step with threshold values of the respective judgment information, respectively, calculating a reward based on a result of the comparison, updating the action merit function based on the calculated reward and the status information acquired in the status information acquisition step after execution of the next production start action instruction, and using the updated action merit function in the action judgment step.
Effects of the invention
According to the production preparation and production start-up work support device, the production preparation and production start-up work support system provided with the same, and the production preparation and production start-up work support method of the present invention, even in a production system such as variant variable production in which a production line or a production process needs to be quickly adjusted for each type of product to be produced, the flow from the first planning of a production preparation work to the final production start-up work can be quickly optimized by machine learning, thereby improving the production QCD and shortening the production preparation time.
Drawings
Fig. 1 is a block diagram showing a configuration of a production preparation and production start-up work support system according to an embodiment of the present invention.
Fig. 2 is an explanatory diagram of the weight setting unit according to the embodiment of the present invention.
Fig. 3 is a signal flow diagram for explaining operations of the production preparation and production start-up work support device according to the embodiment of the present invention.
Fig. 4 is an overall flowchart of the learning process of the production preparation and production start-up work support device according to the embodiment of the present invention.
Fig. 5 is a flowchart of a reward calculation process in the learning process of fig. 4.
Fig. 6 is a step count/round count relationship diagram showing an example of a production start job which can be realized by the production preparation and production start job assisting apparatus according to the embodiment of the present invention.
Fig. 7 is a diagram for explaining a production step in a general production process.
Detailed Description
Next, a production preparation and production start-up work support system according to the present embodiment will be described with reference to the drawings.
Fig. 1 is a block diagram showing a configuration of a production preparation and production start-up work support system according to the present embodiment. The production preparation and production start work support system of the present invention includes a production preparation and production start work support device 100 and a communication unit 8. The production preparation and production start-up work support device 100 is connected to a production line 200 having a plurality of production facilities, for example, and supports a production preparation work of the production line 200. As shown in fig. 1, the production preparation and production start work support apparatus 100 includes a state information acquisition unit 1, an action determination unit 2, a determination information acquisition unit 3, and a learning unit 4.
The status information acquiring unit 1 is connected to the production line 200, and acquires status information including at least a production Material (Material), a production facility (Machine), a work Method (Method), and a worker (Man), which is also referred to as 4M status information, from the production line 200. As an example of the 4M status information, for example, production information such as process information (related to a work method), inventory information (related to a production material), worker information (related to a worker), and equipment information (related to a production equipment) is stored in a production information storage database (a production site IT system) of the production line 200. The process information is information from a first process to a last process performed in time series in accordance with production of a product, the stock information is information including raw materials used for production, stock conditions of the product, and the like, the operator information is information on an operator who actually takes charge of work on the production line and the skill level of the operator, and the equipment information is information on equipment such as the operation capability of the production equipment. Next, an example of specific contents of the 4M status information is illustrated.
As specific contents of the device information of the production device (Machine), for example, at least one of the following information is contained, that is: "device name", "target part number and part name", "processing place and preceding and following steps of a processing target product", "shape, size, weight of raw material", "accuracy of processing and assembling", "performance and function of product", "processing order and processing conditions of a processing target product", "mounting height of a processing target product", "processing time per unit", "stock amount of constituent components", "signal exchange of preceding steps", "restriction conditions of interfering objects and the like", "method of treating problems in existing facilities", "maintenance conditions", "usable floor size and height", "coating color", "power supply voltage, frequency and operating voltage", "gas supply pressure", "availability of cutting oil", "method of treating chips and scraps", "consumables", "installation site", and "contents of accessories".
The job Method information as the job Method (Method) includes, for example, at least one of the following information: "target process", "element work time", "manual work time", "automatic conveyance time", "walking time", "tempo time", "work order", "work content", "focus", "1 cycle time", "observation month and day", "observation time", and "classification number".
As raw Material information of a production Material (Material), for example, at least one of the following information is contained, that is: "name", "specification", "drawing number", "manufacturer", "version number", "use stop date", "part group code", "unit code", "site code", "lead time", "safe stock number", "yield", "minimum lot number", "unit lot number", "maximum lot number", "effective time", "producer", "production date and time", "updater", "update date and time".
The operator information of the operator (Man) includes, for example, at least one of the following information: "person in charge name", "department code", "job", "workday pattern", "mail address", "application start date", "application end date", "update date and time", "skill", "qualification".
The status information acquiring unit 1 may directly acquire the various types of status information by using various detectors, sensors, and the like provided in the production line 200, or may acquire 4M status information from various departments such as a purchasing department (related to production materials), a production department (related to workers and work methods), a quality assurance department (related to work methods), and a production technology department (related to production facilities) via an IT system or the like provided in the production line 200.
The action determining unit 2 is connected to the state information acquiring unit 1, the production line 200, and the learning unit 4 described later, and includes an action merit function storage unit 21, an action determining unit 22, and an action output unit 23. The action merit function storage unit 21 acquires and stores the updated action merit function from the learning unit 4. The action determining unit 22 acquires the 4M status information from the status information acquiring unit 1, and determines the next production start action instruction by the action policy algorithm of reinforcement learning, which will be described later, based on the action cost function read from the action cost function storage unit 21, with reference to the action list stored in the production method/production process storage unit 62 in the action list storage unit 6, which will be described later, based on the acquired 4M status information.
Here, as an example of the type of the production start action instruction, for example, equipment check, equipment inspection, equipment adjustment maintenance, equipment cleaning, equipment oiling, equipment connection fastening, equipment operation condition management, equipment layout optimization, production method optimization, work flow optimization, raw material inspection, component inspection, product inspection, indirect material management standard work, standard time setting, delivery cycle optimization, preparation time optimization, process optimization, technology optimization, worker education, worker skill management, and worker attendance management can be cited. In the learning process described later, the action determination unit 2 may change the production start action instruction in any one or a combination of a plurality of the above-described various categories until the learning end condition is reached.
In addition, although the case where the action determination unit 22 determines the production start action instruction by referring to the various production methods and production processes stored in the production method and production process storage unit 62 has been described here, the present invention is not limited to this, and for example, the various production methods and production processes may be directly stored in the action determination unit 22 and used.
Then, the action output unit 23 outputs the next production start action instruction to the production line 200, so that the production line 200 executes the action instruction. The action output unit 23 may instruct each production facility in the production line 200 to automatically execute the production start action instruction, or may instruct each person in charge of each department such as a purchasing department, a production department, a quality assurance department, and a production technology department to execute the production start action instruction by using a display device.
The judgment information acquiring unit 3 is connected to the production line 200, and acquires judgment information including at least Quality (Quality), Cost (Cost), and Delivery date (Delivery) from the production line 200 after the production line 200 executes the next production start action instruction, and the judgment information is also referred to as QCD judgment information. Next, an example of specific contents of the QCD determination information is illustrated.
The information on the quality (Q) includes, for example, at least one of "defect rate", "straight rate", "number of times of pause due to temporary failure", and "number of device failures". The information on the delivery date (D) includes, for example, at least one of "engineering capacity", "productivity", and "completion amount per unit time". The information on the cost (C) includes, for example, "manufacturing cost". As the information on the quality (Q), the cost (C), and the delivery date (D), for example, "integrated equipment efficiency" is given.
The judgment information acquiring unit 3 may acquire information indicating a delivery date from an MES (Manufacturing Execution System), information indicating whether or not the timing of acquiring a material is stable from a WMS (Warehouse Management System), information indicating whether or not production is possible in a target number of processes from the MES, information indicating whether or not the defective rate is increased from a QMS (Quality Management System), information indicating whether or not the defective rate is increased from the MES or SCADA (Supervisory Control And Data Acquisition And monitoring Control System), information indicating whether or not the required performance is achieved from the MES or SCADA, information indicating how much a temporary failure is suspended, information indicating the difficulty of a processing operation or an assembly process from a PLM (Product life cycle Management System) or a CAD (Computer Aided Design System), or information indicating cost is obtained from information from MES, WMS, or SCADA, etc. as QCD judgment information. The MES, WMS, QMS, SCADA, PLM, CAD and other systems together form the IT system of the production line 200.
The learning unit 4 is connected to the status information acquiring unit 1, the action determining unit 2, and the determination information acquiring unit 3, and includes a reward calculating unit 41 and an action merit function updating unit 42. The reward calculation unit 41 obtains the determination information from the determination information acquisition unit 3, compares the obtained determination information with the threshold value of each determination information, and calculates a reward based on the comparison result. As a specific calculation method of the reward, according to the comparison result of the judgment information and the threshold of each judgment information: increasing the reward when the failure rate, the number of pauses due to temporary failures, the number of equipment failures are below the respective thresholds, or the straight rate is above the thresholds, and otherwise decreasing the reward; increasing the reward if the manufacturing cost associated with cost (C) is below its threshold, and conversely decreasing the reward; increasing the reward in case the engineering capacity, productivity, completion per unit time, etc. related to the delivery date (D) are above the respective threshold value, and conversely decreasing the reward; the reward is increased in the case where the integrated efficiency of the equipment with respect to all of the production quality (Q), the cost (C), and the delivery date (D) is higher than the threshold value thereof, and the reward is decreased in the opposite case. The action merit function update unit 42 acquires the calculated reward from the reward calculation unit 41, acquires the status information after the instruction of the next production start action is executed from the status information acquisition unit 1, updates the action merit function based on the acquired reward and the status information, and supplies the updated action merit function to the action merit function storage unit 21 of the action determination unit 2.
Any learning algorithm may be used as the method for updating the action merit function by the action merit function updating unit 42. As an example, for example, a case where Reinforcement Learning (Reinforcement Learning) is applied is cited. Reinforcement learning is the determination of an action to be taken by an agent (agent) in an environment by observing the current state. The agent obtains a reward from the environment by selecting an action, and learns a countermeasure for obtaining the most reward by a series of actions. As representative methods of reinforcement learning, Q-learning (Q-learning) and TD-learning (time difference learning) are known. For example, in the case of Q learning, a general update (action value table) of the action value function Q (s, a) is represented by equation 1.
[ mathematical formula 1]
In the case of the mathematical formula 1,stindicates the state at time t, atIndicating the action at time t. Due to action atThe state is changed to st+1。rt+1Denotes a reward obtained by the change of the state, γ denotes a discount rate, and α denotes a learning coefficient. Here, when the Q learning is applied, the production start action instruction of the next step determined by the action determination unit 22 is the execution action atIs indicated.
In the update shown in equation 1, if the behavior value of the optimal action a at time t +1 is greater than the action value Q of the action a executed at time t, the action value Q at time t is increased, and conversely, the action value Q at time t is decreased. In other words, the action a at the time ttThe action value function Q (s, a) is updated so that the action value Q(s) approaches the optimum action value at the time t + 1. Thus, the best action value in a certain environment is propagated to the action values in its previous environment in turn.
According to the above configuration, the production preparation and production start work support device of the present invention includes the state information acquiring unit 1, the action judging unit 2, the judgment information acquiring unit 3, and the learning unit 4, and can quickly acquire the action merit function according to the actual situation of the productivity 200 by using machine learning, thereby taking the optimum action according to the state of the productivity 200, and therefore, even when facing a production system such as variant variable production, the flow can be quickly optimized, the production QCD can be improved, and the production preparation time can be shortened.
The main configuration of the production preparation and production start-up work assisting apparatus 100 according to the present embodiment is described above. However, as shown in fig. 1, the production preparation and production start-up work assisting apparatus 100 may further include a weight setting section 5. The weight setting unit 5 is connected to the reward calculation unit 41 of the learning unit 4, and sets a weight for each piece of determination information so that the reward calculation unit 41 changes the amount of increase or decrease of the reward corresponding to each piece of determination information based on the weight of each piece of determination information. The operation principle of the weight setting unit 5 will be described below with reference to the drawings.
Fig. 2 is an explanatory diagram of the weight setting unit 5. As shown in fig. 2, the weight setting unit 5 may acquire QCD determination information as weight determination data from the determination information acquiring unit 3. The weight setting unit 5 may receive a user command as weight determination data via an input device, not shown. Upon receiving the weight determination data, the weight setting unit 5 sets a weight for each determination information based on the weight determination data.
As example 1 in which a weight is set to each determination information based on QCD determination information, for example, the following manner can be adopted: the level of at least one of the QCDs is determined based on the QCD determination information, and the corresponding weight is determined based on the level of the at least one of the QCDs. Specifically, for example, in the quality (Q) level, when the failure rate, the number of pauses due to temporary failures, the number of device failures, and the like are decreased or the straight-line rate is increased, the weight relating to the quality (Q) is increased according to the degree of decrease or increase, respectively, and the larger the failure rate, the number of pauses due to temporary failures, the number of device failures, and the degree of increase in reward, that is, the weight is increased. In addition, in the aspect of the cost (C), when the manufacturing cost of the product is reduced, the weight relating to the cost (C) is increased according to the degree of reduction, and the amount of increase in the reward, that is, the weight is increased as the degree of reduction of the manufacturing cost is increased. In addition, in the delivery date (D), if the construction capacity, productivity, completion amount per unit time, and the like are improved, the weight relating to the delivery date (D) is increased according to the degree of improvement, and the amount of increase in reward, that is, the weight is increased as the degree of improvement is increased. The above description has been made on the case where the weight increases as the increase degree of any one of QCDs increases. On the other hand, when any one of the QCDs deteriorates, the weight may be increased as the degree of deterioration increases.
In addition, as example 2 in which weights are set for the respective pieces of determination information based on the QCD determination information, for example, the following method can be adopted: the QCD is preset with upper and lower thresholds, and according to the QCD determination information, when the QCD determination information falls within the upper and lower thresholds, a certain fixed weight is set, and when one of the QCD determination information deviates from the upper and lower thresholds, a corresponding weight is determined according to the amount of deviation. Specifically, for example, when the production cycle exceeds 65 seconds, production is delayed, and when the production cycle is less than 55 seconds, the difference from the bottleneck process becomes large, the line balance is broken, and the inter-process stock exceeding the necessity is generated. Therefore, the production cycle is set with the upper and lower thresholds of 65 seconds at maximum and 55 seconds at minimum, and when the production cycle is within the upper and lower thresholds, a fixed weight is set for the judgment information corresponding to the delivery date (D), and the reward increases. Conversely, when the production cycle deviates from the threshold range of the upper limit and the lower limit, the weight relating to the delivery date (D) is determined according to the deviation amount, and the larger the deviation amount is, the larger the amount of increase in the reward, that is, the weight is. Further, there is known a technique of setting upper and lower limits to time-series data of a production cycle on a screen of the time-series data of the production cycle with time as a horizontal axis and data of the production cycle as a vertical axis in a plurality of sections. With this technique, when the time series data deviates from the upper and lower limits within a plurality of intervals, a warning message is displayed to the operator of productivity, and the operator makes a quality determination. In the present embodiment, the time-series data of the determination information may be weighted by using the upper and lower limits set for the time-series data as the upper and lower limit thresholds set by the weight setting unit 5.
In addition, as example 3 in which weights are set for the respective pieces of determination information based on the QCD determination information, for example, the following method can be adopted: thresholds are set in advance for the QCDs, and the weight setting unit 5 determines the weights corresponding to the respective thresholds based on the deviation amounts between the QCD determination information and the respective thresholds. Specifically, for example, thresholds are preset for the QCDs based on the 4M status information, and the weight setting unit 5 sets the weight to be larger as the deviation amount between the QCD determination information and the threshold is larger. Further, when the rate of change in the amount of deviation between the QCD determination information and the threshold value increases, the weight may be set based on the rate of change in the amount of deviation, and the weight setting unit 5 may set the weight to be larger as the rate of change in the amount of deviation is larger. Thus, when the amount of deviation increases in an acceleration manner, a larger weight can be set.
In addition, the weight setting manners of the above-described examples 2 and 3 may be combined, and the threshold value may be set in the vicinity of the upper limit within the range of the upper limit and the lower limit. Specifically, for example, since the production cycle varies greatly depending on the process, the production cycle may exceed the upper limit in the vicinity of the upper limit within the range between the upper limit and the lower limit. To avoid such a risk, it is preferable to set the threshold value at a position close to the upper limit above the lower limit. Thus, the production cycle can be stabilized in the vicinity of the lower limit within the range of the upper limit and the lower limit, and the probability that the production cycle exceeds the upper limit can be reduced. Thus, a plurality of examples 1 to 3 described above can be used in combination.
As an example of setting the weight to each judgment information based on the user instruction, as shown in fig. 2, the operator as the user may determine the weight to be given to each judgment information based on, for example, the supply and demand status of the product to be produced or 4M status information acquired from an external device such as a display connected to the production preparation and production start-up work assisting apparatus 100 via the communication unit 8 described later, and input the weight to the weight setting unit 5 via an input device not shown. For example, when the user determines that only one or several items of the determination information need to be compared with the threshold value according to the supply and demand situation, the 4M status information, and the like, the weight of the determination information that does not need to be compared may be set to 0. Thus, the user can concentrate on the optimization of one or more aspects of the production line according to the needs of the user, thereby further reducing the burden of production preparation and production starting operation and shortening the time of the production preparation and production starting operation.
Then, the weight setting unit 5 sets a weight for each piece of judgment information, and stores each weight in the weight storage unit 51 to be used by the learning unit 4 in calculating the reward.
As shown in fig. 1, the production preparation and production start work support apparatus 100 may further include an action list storage unit 6, and the action list storage unit 6 is connected to the action determination unit 22, and includes an initial production start information providing unit 61 in addition to the production method/production process storage unit 62. The initial production start information providing unit 61 stores an initial production start action instruction, which is a list of actions to be executed in an initial stage of learning, and provides the initial production start action instruction to the action determining unit 22 of the action determining unit 2 in an initial stage after the start of learning. As an example of a method of determining the initial production start action instruction, for example, a production technician as a user may input the instruction through an input device not shown, or the initial production start action instruction may be determined by referring to the action history up to now stored in the initial production start information providing unit 61 in advance.
In the initial stage after the learning starts, the action judging section 2 first causes the production line 200 to execute the initial production start action instruction, the judgment information acquiring section 3 acquires the judgment information after the production line 200 has executed the initial production start action instruction, and the learning section 4 updates the action merit function based on the comparison result of the judgment information and the state information after the execution of the initial production start action instruction, and resupplies the updated action merit function to the action judging section 2.
As shown in fig. 1, the production preparation and production start-up work assisting apparatus 100 may further include a circulation control unit 7. The loop control unit 7 controls the loop operation of the state information acquisition unit 1, the action determination unit 2, the determination information acquisition unit 3, and the learning unit 4, and includes a step management unit 71 and a round management unit 72. The step management section 71, when the round end condition is satisfied, causes the loop of one round to be ended and the next round to be entered. The round management part 72 causes the learning to be ended if the learning end condition is satisfied. The specific modes of the step management and the round management will be described together in the following description of the learning process.
As shown in fig. 1, the production preparation and production start-up work support device 100 according to the present embodiment may constitute a production preparation and production start-up work support system together with the communication unit 8. The communication unit 9 is connected to the status information acquiring unit 1, the action determining unit 2, the determination information acquiring unit 3, and the learning unit 4, respectively, and can transmit the status information, the determination information, the next production start action instruction, and the action merit function to an external device such as a display or a cloud server connected to a production preparation and production start work support system via a network. The communication unit 9 can transmit various information related to the learning of the action value function to an external device, thereby facilitating real-time monitoring, operation inspection, failure diagnosis, system maintenance improvement, and the like of the entire production preparation and production start-up work support system by an operator, a manufacturer, and the like.
Next, the operation principle of the production preparation and production start-up work assisting apparatus 100 will be described with reference to fig. 3 to 6. Fig. 3 is a signal flow diagram for explaining the operation of the production preparation and production start-up work assisting apparatus 100 according to the present embodiment. Fig. 4 is an overall flowchart of the learning process of the production preparation and production start-up work support device 100 according to the present embodiment. Fig. 5 is a flowchart of a reward calculation process in the learning process of fig. 4. Fig. 6 is a step count/round count relationship diagram showing an example of a production start job which can be realized by the production preparation and production start job assisting apparatus 100 according to the present embodiment.
As shown in fig. 3, the production line 200 corresponds to an "environment" in reinforcement learning; the state information acquiring unit 1, the action determining unit 2, the determination information acquiring unit 3, and the learning unit 4 constitute an "agent" in reinforcement learning. The parameters used for learning are set by the initial production start information providing unit 61, the production method/production process storage unit 62, and the weight setting unit 5.
As shown in fig. 3 and 4, when the learning of the production preparation and production start-up work is started, the action determination unit 2 first acquires an initial production start action instruction from the initial production start information provision unit 61. The method for determining the initial production start action indication is as described above. Next, a round start instruction is issued by the round managing unit 72 in the cycle control unit 7, not shown, and a step start instruction is issued by the step managing unit 71.
Then, the action judging section 2 selects the initial production start action instruction as an action, and causes the production line 200 to execute the action. After the initial production start action instruction is executed on the production line 200, the state information acquisition unit 1 acquires the 4M state information of the next state from the production line 200, and the reward calculation unit 41 of the learning unit 4 calculates the reward from the QCD determination information acquired from the production line 200 by the determination information acquisition unit 3 after the initial production start action instruction is executed. The specific flow of the reward calculation will be described below. Next, the action-value-function updating unit 42 stores the reward calculated by the reward calculating unit 41 in a memory, not shown, as experience, updates the action value function based on the calculated reward and the 4M status information of the next status from the status-information acquiring unit 1, and transmits the updated action value function to the action-value-function storage unit 21 in the action determining unit 2.
Then, the step management unit 71 determines whether or not the round end condition is satisfied. Here, the round end condition refers to a case where any one of the QCD determination information is deviated from a predetermined condition value as a result of comparison by the reward calculation unit 41 of the learning unit 4, or a case where the number of steps of the cyclic operation during learning exceeds a predetermined upper limit of the number of steps. The QCD determination information is not equal to or greater than a predetermined condition value, and may be different from or greater than a predetermined condition value.
When the step management unit 71 determines that the round end condition is not satisfied ("no is the result of determination" is satisfied "), the flow returns to the step start stage, and the action determination unit 2 selects the next production start action instruction as an action based on the latest action cost function stored in the action cost function storage unit 21 and the current 4M status information from the status information acquisition unit 1, and causes the production line 200 to execute the action, thereby starting to repeat the learning step.
On the other hand, when the step management unit 71 determines that the above-described round end condition is satisfied ("yes" determination result is satisfied. Here, the learning end condition is a case where the number of rounds of the cyclic operation during learning exceeds a predetermined upper limit of the number of rounds.
If the round management unit 72 determines that the learning end condition is not satisfied ("does learning end condition satisfied", determination result is no), the flow returns to the round start stage, and learning of the next round is resumed. In this case, the learning process of the above steps is repeatedly performed.
On the other hand, in the case where the round management part 72 determines that the above-described learning end condition is satisfied ("yes" determination result is satisfied.
Next, a specific method for the action determining unit to determine the next production start action instruction by using the action policy algorithm of reinforcement learning will be described. Here, the epsilon-greedy policy will be described as an example. In the learning initial stage, due to inexperience, the next production start action indication with the maximum bid value may not be immediately judged. In this case, an action may be selected at random at a search rate of ∈ 0.1 by using an ∈ -greedy policy, and an action having the greatest value may be selected among the other actions selected by greedy. Then, ε is gradually decreased, eventually converging the algorithm. For example, in the learning start stage, the initial production start information providing unit 61 may give initial values to various action conditions in advance, and then change at least one 4M condition within a range to start the cycle. Specifically, for example, the cycle time of production may be reduced by 5 seconds from 65 seconds, which is an initial value, or the safety stock ratio may be reduced by 5% to produce, and the combination of 4M and the action that optimizes the QCD within a predetermined range may be learned by acquiring and cycling the parameters at that time. In addition to the epsilon-greedy strategy, other learning-intensive action strategy algorithms such as genetic algorithm may be used to develop learning by randomly selecting an action with a certain probability.
The following describes a method of reward calculation in the learning process of fig. 4. As shown in fig. 5, after the reward calculation is started, first, in step S101, the reward calculation unit 41 in the learning unit 4 acquires QCD determination information from the determination information acquisition unit 3, and proceeds to step S102. In step S102, the reward calculation unit 41 compares the determination information on the quality (Q) in the QCD determination information with a preset quality threshold (Q threshold). In the present embodiment, a description will be given of a defect rate as an example of the determination information on the quality (Q). When the failure rate is higher than the quality threshold (step S102: high), the process proceeds to step S103, and the reward calculation unit 41 reduces the reward by the weight set by the weight setting unit 5 in the weight setting manner described above, for example, -10, and proceeds to step S106. When the defective rate is equal to the quality threshold (step S102: same), the process proceeds to step S104, and the reward calculation section 41 keeps the reward unchanged, and proceeds to step S106. When the defective rate is lower than the quality threshold (step S102: low), the process proceeds to step S105, and the reward calculation unit 41 increases the reward by the weight set by the weight setting unit 5 in the weight setting method described above, for example, by +10, and proceeds to step S106.
In step S106, the reward calculation unit 41 compares the determination information on the cost (C) in the QCD determination information with a preset cost threshold (C threshold). In the present embodiment, the manufacturing cost of a product is described as an example of the determination information on the cost (C). When the manufacturing cost is higher than the cost threshold (step S106: high), the process proceeds to step S107, and the reward calculation unit 41 reduces the reward by the weight set by the weight setting unit 5 in the weight setting manner described above, for example, -3, and proceeds to step S110. When the manufacturing cost is equal to the cost threshold (step S106: same), the process proceeds to step S108, and the consideration calculating section 41 leaves the consideration unchanged, and proceeds to step S110. When the manufacturing cost is lower than the cost threshold (step S106: low), the process proceeds to step S109, and the reward calculation unit 41 adds a reward, for example +3, with the weight set by the weight setting unit 5 in the weight setting method described above, and proceeds to step S110.
In step S110, the reward calculation unit 41 compares the determination information on the delivery date (D) in the QCD determination information with a delivery date threshold value (D threshold value) set in advance. In the present embodiment, the completion amount per unit time is explained as an example of the judgment information on the delivery date (D). When the completion amount per unit time is lower than the delivery date threshold (step S110: low), the process proceeds to step S111, and the reward calculation unit 41 reduces the reward by the weight set by the weight setting unit 5 in the weight setting manner described above, for example, -5, and then ends the reward calculation process, and sends the calculation result to the action value function update unit 42. When the completion amount per unit time is equal to the delivery date threshold value (step S110: same), it proceeds to step S112, and the reward calculation section 41 keeps the reward unchanged, and then ends the reward calculation process, and sends the calculation result to the action value function updating section 42. When the completion amount per unit time is higher than the delivery date threshold (step S110: high), the process proceeds to step S113, and the reward calculation section 41 increases the reward by the weight set by the weight setting section 5 in the weight setting manner described above, for example, by +5, and then ends the reward calculation process, and sends the calculation result to the action value function update section 42.
Although not shown in fig. 4 and 5, the weight setting step may be performed before the reward calculation is started. In this weight setting step, the weight setting unit 5 sets a weight for each piece of judgment information in the various manners described above, and transmits the set weight to the reward calculation unit 41. Thus, in the reward calculation process, the reward calculation unit 41 can change the amount of increase or decrease of the reward corresponding to each determination information, based on the weight of each determination information set.
Next, an example of the production start-up work that can be performed by the production preparation and production start-up work assisting apparatus 100 according to the present embodiment will be described with reference to fig. 6.
In the step number/round number relationship diagram of fig. 6, the horizontal axis represents the number of rounds experienced during learning, and the vertical axis represents the number of steps experienced during learning (e.g., the number of products, time, etc. for completing continuous production).
In this example, the entire process of completing production of 1 product from the first production facility to the last production facility is 1 step, and when it is determined that the QCD of the product that completed production is better than the predetermined QCD threshold value, the production of the 2 nd product is performed and the process proceeds to the next 1 step. When it is determined that the QCD of the product whose production is completed deviates from the preset condition value, or when the number of repeated steps exceeds the preset upper limit of the number of steps (for example, 200 steps, that is, 200 products are continuously produced), the production of 1 round is ended and the next 1 round is entered. In addition, when the number of repeated rounds exceeds a preset upper limit of the number of rounds (for example, 1000 rounds), the learning process is ended.
As shown in the graph of fig. 6, in the present example, the number of continuous productions completed in the vicinity of more than 400 rounds from the start of learning exceeds 100, and stable production is realized. As described above, the production preparation and production start-up work assisting apparatus 100 according to the present embodiment can optimize the production preparation and production start-up work process and quickly realize stable production.
The preferred embodiments of the present invention have been described above. According to the production preparation and production start-up work support device, system and method of the present embodiment, the next production start-up action instruction is determined from the state information based on the action cost function, the judgment information is acquired after the production line executes the next production start-up action instruction, the judgment information is compared with the threshold value of each judgment information, the reward is calculated based on the comparison result, the action cost function is updated based on the calculated reward and the state information acquired by the state information acquisition unit after the production start-up action instruction of the next step is executed, and therefore, the optimal operation condition of the production line and the start-up work of the production line are calculated without depending on the experience of the technician who has a difference in technical level, experience and the like in the production preparation and production start-up work stages, therefore, it is possible to reduce the time taken to calculate the optimum operating conditions, to reduce the quality variation of the calculated optimum operating conditions, and to ensure that each production line can be started up with the same standard of QCD. Especially in variant production, the production of QCD can be improved and the production setup time can be shortened.
Further, by setting a weight to each piece of judgment information and changing the amount of increase or decrease of the reward corresponding to each piece of judgment information based on the weight of each piece of judgment information, it is possible to appropriately adjust the degree of influence of each of the quality (Q), the cost (C), and the delivery date (D) on the production process in accordance with the degree of influence of each of the quality (Q), the cost (C), and the delivery date (D) on the production process in the actual production preparation work, or the degree of influence of each of the quality (Q), the cost (C), and the delivery date (D) on the reward in various 4M states or the demand of the user himself/herself, and therefore, it is possible to further optimize the operating conditions of the production line in the production preparation and production start-up work and obtain a better plan of the production preparation and production start-up work.
It should be noted that all aspects of the embodiments disclosed herein are merely exemplary and not restrictive. The scope of the present invention is indicated by the appended claims, rather than the foregoing embodiments, and all changes and modifications that come within the meaning and range of equivalency of the claims are intended to be embraced therein.
Industrial applicability of the invention
As described above, the production preparation and production start-up work assisting apparatus, the production preparation work assisting system provided with the production preparation and production start-up work assisting apparatus, and the production preparation work assisting method according to the present invention are useful for assisting the production preparation and production start-up stage.
Description of the reference symbols
1 status information acquisition unit
2 action judging part
3 judgment information acquisition unit
4 learning part
5 weight setting unit
6 action list storage part
7 cycle control part
8 communication part
21 action value function storage part
22 action determining part
23 action output part
41 reward calculating part
42 action value function updating part
51 weight storage unit
61 initial production start information providing part
62 production mode/production process storage unit
71 step management part
72 round management part
100 production preparation and production start-up operation auxiliary device
200 production line
Claims (15)
1. A production preparation and production start-up work assisting device that assists production preparation and production start-up work of a production line having a plurality of production apparatuses, comprising:
a state information acquisition unit that acquires state information including at least a production material, a production facility, a work method, and a worker from the production line;
an action judging section for determining a next production start action instruction by using an action strategy algorithm of reinforcement learning based on the state information from the state information acquiring section based on an action cost function, and causing the production line to execute the next production start action instruction;
a judgment information acquisition unit that acquires judgment information including at least quality, cost, and delivery date from the production line after the production line executes the next production start action instruction; and
and a learning unit that compares the determination information from the determination information acquisition unit with each of the threshold values of the determination information, calculates a reward based on a result of the comparison, updates the action merit function based on the calculated reward and the status information acquired by the status information acquisition unit after execution of the next production start action instruction, and provides the updated action merit function to the action determination unit.
2. The production preparation and production start-up work assistance device according to claim 1,
further comprising a weight setting unit for setting a weight for each of the determination information,
the learning unit changes an amount of increase or decrease of the reward corresponding to each of the determination information based on the weight of each of the determination information set by the weight setting unit.
3. The production preparation and production start-up work assisting apparatus according to claim 2,
the weight setting unit sets the weight to the determination information based on the determination information from the determination information acquiring unit.
4. The production preparation and production start-up work assisting apparatus according to claim 2,
the weight setting unit sets the weight to the determination information based on a user instruction.
5. The production preparation and production start-up work assisting device according to any one of claims 1 to 4,
further comprising an initial production start information providing unit for determining an initial production start action instruction based on a user input or an action history stored in the initial production start information providing unit and providing the initial production start action instruction to the action judging unit,
the action judging unit causes the production line to execute the initial production start action instruction,
the judgment information acquisition unit acquires the judgment information from the production line after the initial production start action instruction is executed by the production line,
the learning unit compares the determination information from the determination information acquisition unit with each of the threshold values of the determination information, calculates a reward based on a result of the comparison, updates the action merit function based on the calculated reward and the status information acquired by the status information acquisition unit after the initial production start action instruction is executed, and provides the updated action merit function to the action determination unit.
6. The production preparation and production start-up work assisting device according to any one of claims 1 to 4,
further comprises a cycle control part for controlling the cycle action of the state information acquisition part, the action judgment part, the judgment information acquisition part and the learning part, and comprises a step management part and a round management part,
the step management unit ends the cycle of one round and advances to the next round when the comparison result of the learning unit indicates that any one of the determination information is deviated from a predetermined condition value or when the number of steps of the cyclic operation exceeds a predetermined upper limit of the number of steps,
the round management unit ends learning when the number of rounds of the cyclic operation exceeds a predetermined upper limit.
7. The production preparation and production start-up work assisting device according to any one of claims 1 to 4,
the types of the production starting action instructions comprise any one or more of equipment point inspection, equipment adjustment and maintenance, equipment cleaning, equipment oiling, equipment connecting piece fastening, equipment running condition management, equipment layout optimization, generation method optimization, processed product circulation mode optimization, raw material inspection, component inspection, product inspection, indirect material management standard operation, standard time setting, delivery cycle optimization, preparation time optimization, process optimization, technology optimization, operator education, operator skill management and operator attendance rate management.
8. A production preparation and production start-up work assistance system, comprising:
the production preparation and production start-up work assisting apparatus according to any one of claims 1 to 7; and
and a communication unit that transmits the state information, the determination information, the next production start action instruction, and the action cost function to an external device connected to the production preparation and production start operation support system via a network.
9. A production preparation and production start-up work assisting method for assisting production preparation and production start-up work of a production line having a plurality of production devices, comprising:
a state information acquisition step of acquiring state information including at least a production material, production equipment, an operation method, and an operator from the production line;
an action judgment step of determining a next production start action instruction by using an action policy algorithm of reinforcement learning based on an action cost function according to the state information acquired in the state information acquisition step, and causing the production line to execute the next production start action instruction;
a judgment information acquisition step of acquiring judgment information including at least quality, cost, and delivery date from the production line after the production line executes the next production start action instruction; and
a learning step of comparing the judgment information acquired in the judgment information acquisition step with a threshold value of each of the judgment information, respectively, calculating a reward based on a result of the comparison, updating the action merit function in accordance with the calculated reward and the status information acquired in the status information acquisition step after execution of the production start action instruction of the next step, and using the updated action merit function in the action judgment step.
10. The production preparation and production start-up work assisting method according to claim 9,
further comprising a weight setting step of setting a weight for each of the judgment information,
in the learning step, an amount of increase or decrease of the reward corresponding to each of the determination information is changed in accordance with the weight of each of the determination information set in the weight setting step.
11. The production preparation and production start-up work assisting method according to claim 10,
in the weight setting step, the weight is set to the determination information based on the determination information acquired in the determination information acquisition step.
12. The production preparation and production start-up work assisting method according to claim 10,
in the weight setting step, the weight is set to the determination information based on a user instruction.
13. The production preparation and production start-up work assisting method according to any one of claims 9 to 12,
further comprising an initial production start information providing step of determining and providing an initial production start action instruction based on user input or the stored action history,
in the action judging step, the production line is caused to execute the initial production starting action instruction,
in the judgment information acquisition step, the judgment information is acquired from the production line after the initial production start action instruction is executed by the production line,
in the learning step, the judgment information acquired in the judgment information acquisition step is compared with each of the threshold values of the judgment information, a reward is calculated based on a result of the comparison, the action merit function is updated based on the calculated reward and the status information acquired in the status information acquisition step after the initial production start action instruction is executed, and the updated action merit function is used in the action judgment step.
14. The production preparation and production start-up work assisting method according to any one of claims 9 to 12,
further comprising a loop control step of controlling loop operations of the state information acquisition step, the action judgment step, the judgment information acquisition step, and the learning step, including a step management step and a round management step,
in the step management step, when the comparison result in the learning step is that any one of the determination information is deviated from a predetermined condition value or when the number of steps of the cyclic operation exceeds a predetermined upper limit of the number of steps, the cycle of one round is ended and the next round is entered,
in the round managing step, in a case where the number of rounds of the cyclic action exceeds a predetermined upper limit of the number of rounds, the learning is caused to end.
15. The production preparation and production start-up work assisting method according to any one of claims 9 to 12,
the types of the production starting action instructions comprise any one or more of equipment point inspection, equipment adjustment and maintenance, equipment cleaning, equipment oiling, equipment connecting piece fastening, equipment running condition management, equipment layout optimization, generation method optimization, processed product circulation mode optimization, raw material inspection, component inspection, product inspection, indirect material management standard operation, standard time setting, delivery cycle optimization, preparation time optimization, process optimization, technology optimization, operator education, operator skill management and operator attendance rate management.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911075243.0A CN112766530A (en) | 2019-11-06 | 2019-11-06 | Production preparation and production start operation auxiliary device, system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911075243.0A CN112766530A (en) | 2019-11-06 | 2019-11-06 | Production preparation and production start operation auxiliary device, system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112766530A true CN112766530A (en) | 2021-05-07 |
Family
ID=75692805
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911075243.0A Pending CN112766530A (en) | 2019-11-06 | 2019-11-06 | Production preparation and production start operation auxiliary device, system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112766530A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113507514A (en) * | 2021-06-30 | 2021-10-15 | 歌尔股份有限公司 | Multi-station control method, production line, production system and readable storage medium |
| CN113835405A (en) * | 2021-11-26 | 2021-12-24 | 阿里巴巴（中国）有限公司 | Generation method, device and medium for balance decision model of garment sewing production line |
| CN117973635A (en) * | 2024-03-28 | 2024-05-03 | 中科先进(深圳)集成技术有限公司 | Decision prediction method, electronic device, and computer-readable storage medium |
| CN117973635B (en) * | 2024-03-28 | 2024-06-07 | 中科先进(深圳)集成技术有限公司 | Decision prediction method, electronic device, and computer-readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20090000285A (en) * | 2007-02-15 | 2009-01-07 | 주식회사 엠티에스코리아 | A process control system and a method thereof |
| CN105427021A (en) * | 2015-10-30 | 2016-03-23 | 江苏云道信息技术有限公司 | Intelligent clothes production scheduling method |
| CN107045330A (en) * | 2016-02-09 | 2017-08-15 | 发那科株式会社 | Production control system and comprehensive production control system |
| CN107767760A (en) * | 2016-08-23 | 2018-03-06 | 发那科株式会社 | Rote learning device, robot system and learning by rote |
| CN108693851A (en) * | 2017-03-31 | 2018-10-23 | 发那科株式会社 | Behavioural information learning device, robot control system and behavioural information learning method |
-
2019
- 2019-11-06 CN CN201911075243.0A patent/CN112766530A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20090000285A (en) * | 2007-02-15 | 2009-01-07 | 주식회사 엠티에스코리아 | A process control system and a method thereof |
| CN105427021A (en) * | 2015-10-30 | 2016-03-23 | 江苏云道信息技术有限公司 | Intelligent clothes production scheduling method |
| CN107045330A (en) * | 2016-02-09 | 2017-08-15 | 发那科株式会社 | Production control system and comprehensive production control system |
| CN107767760A (en) * | 2016-08-23 | 2018-03-06 | 发那科株式会社 | Rote learning device, robot system and learning by rote |
| CN108693851A (en) * | 2017-03-31 | 2018-10-23 | 发那科株式会社 | Behavioural information learning device, robot control system and behavioural information learning method |
Non-Patent Citations (2)
| Title |
|---|
| MAHADEVAN S, THEOCHAROUS G.: "Optimizing production manufacturing using reinforcement learning", FLAIRS CONFERENCE, pages 332 - 337 * |
| 秦蕊, 曾帅, 李娟娟等: "基于深度强化学习的平行企业资源计划", 自动化学报, vol. 43, no. 9, pages 1588 - 1596 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113507514A (en) * | 2021-06-30 | 2021-10-15 | 歌尔股份有限公司 | Multi-station control method, production line, production system and readable storage medium |
| CN113507514B (en) * | 2021-06-30 | 2024-04-30 | 歌尔股份有限公司 | Multi-station control method, production line, production system and readable storage medium |
| CN113835405A (en) * | 2021-11-26 | 2021-12-24 | 阿里巴巴（中国）有限公司 | Generation method, device and medium for balance decision model of garment sewing production line |
| CN113835405B (en) * | 2021-11-26 | 2022-04-12 | 阿里巴巴（中国）有限公司 | Generation method, device and medium for balance decision model of garment sewing production line |
| CN117973635A (en) * | 2024-03-28 | 2024-05-03 | 中科先进(深圳)集成技术有限公司 | Decision prediction method, electronic device, and computer-readable storage medium |
| CN117973635B (en) * | 2024-03-28 | 2024-06-07 | 中科先进(深圳)集成技术有限公司 | Decision prediction method, electronic device, and computer-readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9958861B2 (en) | Production control system and integrated production control system | |
| EP3214509B1 (en) | Control parameter automatic-adjustment apparatus, control parameter automatic-adjustment method, and control parameter automatic-adjustment apparatus network | |
| US8321187B2 (en) | Process simulation utilizing component-specific consumption data | |
| US8126574B2 (en) | System and method for dynamic multi-objective optimization of machine selection, integration and utilization | |
| KR100333454B1 (en) | Production estimate management system | |
| US10733577B2 (en) | Preventive maintenance management system and method for generating maintenance schedule of machine, and cell controller | |
| EP3588400B1 (en) | Operational plan optimization device and operational plan optimization method | |
| KR102171807B1 (en) | System for predicting product failure in process and a method for generating learning model for failure prediction | |
| CN112766530A (en) | Production preparation and production start operation auxiliary device, system and method | |
| JP2004086911A (en) | Method and system for improving process management, and storage medium | |
| CN112150435A (en) | Construction management method and device for prefabricated part | |
| JP2017215680A (en) | Production controller for controlling production cell implementing maintenance work | |
| JP6880343B1 (en) | Scheduler system, scheduler management device and machine learning device | |
| KR100453699B1 (en) | Method and apparatus of providing production control system with manufacturing process monitoring system | |
| EP4138008A1 (en) | Energy supply/demand operation guidance device and method for energy supply/demand operation in ironworks | |
| JP2023074027A (en) | Production planning system and production planning method | |
| KR20230081029A (en) | Method for scheduling of bearing manufacturing operations | |
| JPH07129687A (en) | Production planning alarm system | |
| JP2007156872A (en) | Management method of production process | |
| JP6984786B1 (en) | System construction support program, system construction support method, and system construction support device | |
| WO2024089771A1 (en) | Equipment maintenance management system | |
| JP2024054882A (en) | Failure prediction device | |
| JP6894414B2 (en) | Inventory management support system and method | |
| JP2957561B1 (en) | Production line management method | |
| Zhou et al. | A framework for effective shop floor control in wafer fabs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |