JP7189777B2 - Metal material production system and metal material production method - Google Patents

Metal material production system and metal material production method Download PDF

Info

Publication number
JP7189777B2
JP7189777B2 JP2019001383A JP2019001383A JP7189777B2 JP 7189777 B2 JP7189777 B2 JP 7189777B2 JP 2019001383 A JP2019001383 A JP 2019001383A JP 2019001383 A JP2019001383 A JP 2019001383A JP 7189777 B2 JP7189777 B2 JP 7189777B2
Authority
JP
Japan
Prior art keywords
unit
production
metal material
condition
learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2019001383A
Other languages
Japanese (ja)
Other versions
JP2020112882A (en
Inventor
岳己 磯松
正靖 笠原
勇 曹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
THE FURUKAW ELECTRIC CO., LTD.
Original Assignee
THE FURUKAW ELECTRIC CO., LTD.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by THE FURUKAW ELECTRIC CO., LTD. filed Critical THE FURUKAW ELECTRIC CO., LTD.
Priority to JP2019001383A priority Critical patent/JP7189777B2/en
Publication of JP2020112882A publication Critical patent/JP2020112882A/en
Application granted granted Critical
Publication of JP7189777B2 publication Critical patent/JP7189777B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Description

本発明は、金属材料生産部と機械学習部とを有する金属材料生産システムおよび金属材料生産方法に関し、特に、金属材料生産部を構成する複数の設備において、各設備の生産条件を、熟練オペレータ（人）による調整を行なわなくても、常に最適な生産条件で安定して金属材料を製造することができる金属材料生産システムを実現する。 The present invention relates to a metal material production system and a metal material production method having a metal material production department and a machine learning department. To realize a metal material production system capable of stably manufacturing a metal material always under optimum production conditions without adjustment by a person.

金属材料を製造するには、金属材料に求められる機械特性、電気的特性、形状、表面状態などの品質に応じて、その製造時における複数の設備に無数に存在する生産条件の中から、１つの生産条件を選択して製造する必要がある。また、その材質、幅、板厚などの設計も考慮して、生産条件（例えば圧延、熱処理、表面処理時などの条件）を決定する必要がある。したがって、製造する金属材料ごとに、生産条件の最適な値を決定する条件出し作業を行う必要がある。しかしながら、各工程の最適な生産条件を見出すには、熟練オペレータ（人）が、経験に基づいて加工後の形状や表面状態を、検出器や目視にて状況を確認しながら各種操作条件を調整し、最適になるよう操作する必要がある。そのため、オペレータが時間をかけて最適の生産条件を決定する必要がある。また、経験の少ないオペレータが、熟練オペレータのように各設備の最適な生産条件を決定できるようになるには、長期間の教育が必要であった。 In order to manufacture a metal material, according to the quality such as mechanical properties, electrical properties, shape, surface condition, etc. required for the metal material, one of the innumerable production conditions that exist in multiple facilities at the time of manufacturing is selected. It is necessary to select one production condition for manufacturing. In addition, it is necessary to determine production conditions (for example, rolling, heat treatment, surface treatment conditions, etc.) in consideration of the design of the material, width, plate thickness, and the like. Therefore, it is necessary to determine the optimal production conditions for each metal material to be manufactured. However, in order to find the optimum production conditions for each process, a skilled operator (person) adjusts various operating conditions while checking the shape and surface condition after processing based on experience with a detector or visually. and should be manipulated to optimize it. Therefore, it is necessary for the operator to spend time determining the optimum production conditions. In addition, long-term training is required for inexperienced operators to be able to determine the optimum production conditions for each facility like skilled operators.

オペレータによる最適操作条件の条件出し作業を軽減するための従来技術として、一般に、検出器で検知した数値が規定を外れるとアラームが鳴る技術や、検出器で検出した表面欠陥部のサイズや位置をモニターで示す技術がある。 As a conventional technology to reduce the operator's task of determining the optimum operating conditions, generally, there is a technology that sounds an alarm when the numerical value detected by the detector deviates from the regulation, and a technology that detects the size and position of the surface defect detected by the detector. There is technology to show on the monitor.

また、特許文献１は、射出成形機の条件出し作業において、成形条件を変更した際にその変更した条件の履歴を保持しておき、適切な成形を達成していた特定の時点における過去の成形条件を再生する場合において、その変更履歴を現在から特定の時点の時点まで遡って読み出す技術が開示されている。このような技術によれば、過去の射出成形条件をデータの容量を小さくして、容易に再生することができる。 In addition, Patent Document 1 discloses that when the molding conditions are changed in the condition setting work of the injection molding machine, the history of the changed conditions is retained, and the past molding at a specific time when appropriate molding was achieved. A technique is disclosed for reading the change history of conditions retroactively from the present to a specific point in time when reproducing the conditions. According to such a technique, past injection molding conditions can be easily reproduced by reducing the volume of data.

特開平１１－３３３８９９号公報JP-A-11-333899

以上のような技術では、条件出し作業を軽減し得るものではあるが、作業を行うオペレータの技術レベルによっては、最適な操作条件を算出するまでに長時間を要することがある。また、複数のオペレータ同士でも、それぞれ最適な操作条件にずれ（差）が生じることがあり、オペレータが異なる場合には、同じ操作条件で同様の品質が担保されないこともある。 Although the above-described technique can reduce the work of determining conditions, it may take a long time to calculate the optimum operating conditions depending on the skill level of the operator who performs the work. In addition, even among a plurality of operators, the optimum operating conditions may differ (differences), and if the operators are different, the same operating conditions may not guarantee the same quality.

本発明は、以上のような実情に鑑みなされたものであり、特に、金属材料生産部を構成する複数の設備において、各設備の生産条件を、熟練オペレータ（人）による調整を行なわなくても、常に最適な生産条件で安定して金属材料を製造することができる金属材料生産システムおよび金属材料生産方法を提供することを目的とする。 The present invention has been made in view of the above circumstances, and in particular, in a plurality of facilities constituting a metal material production department, the production conditions of each facility can be adjusted without adjustment by a skilled operator (person). An object of the present invention is to provide a metal material production system and a metal material production method that can always stably produce metal materials under optimum production conditions.

本発明者らは、上述した課題を解決すべく鋭意検討を重ねた。その結果、金属材料生産部と機械学習部とを有する金属材料生産システムであって、前記機械学習部は、前記金属材料生産部において実行中の金属材料生産に関する物理量を観測する状態観測部［Ａ］と、前記状態観測部［Ａ］で観測した前記物理量を、データとして記憶する物理量データ記憶部［Ｂ］と、機械学習における報酬条件を設定する報酬条件設定部［Ｃ］と、前記状態観測部［Ａ］で観測した前記物理量のデータ、および前記報酬条件設定部［Ｃ］で設定された前記報酬条件に基づいて報酬を算出する報酬計算部［Ｄ］と、前記報酬計算部［Ｄ］で算出した前記報酬、前記物理量データ、および前記金属材料生産部で設定されている生産条件に基づいて生産条件調整の機械学習を行う生産条件学習部［Ｅ］と、前記生産条件学習部［Ｅ］で機械学習した学習結果を記憶する学習済み条件記憶部［Ｆ］と、前記生産条件学習部［Ｅ］での前記学習結果に基づいて、前記金属材料生産部を構成する各設備の生産条件の調整量を決定して出力する生産条件出力部［Ｇ］と、を備えることにより、特に、金属材料生産部を構成する複数の設備において、各設備の生産条件を、熟練オペレータ（人）による調整を行なわなくても、常に最適な生産条件で安定して金属材料を製造することができる金属材料生産システムを提供することができることを見出し、本発明を完成するに至った。 The present inventors have made extensive studies to solve the above-described problems. As a result, a metal material production system having a metal material production unit and a machine learning unit, wherein the machine learning unit includes a state observation unit [A ], a physical quantity data storage unit [B] that stores the physical quantity observed by the state observation unit [A] as data, a reward condition setting unit [C] that sets reward conditions in machine learning, and the state observation unit [B] a reward calculation unit [D] for calculating a reward based on the data of the physical quantity observed in the unit [A] and the reward condition set by the reward condition setting unit [C]; and the reward calculation unit [D]. A production condition learning unit [E] that performs machine learning for adjusting production conditions based on the remuneration calculated in , the physical quantity data, and the production conditions set in the metal material production unit; and the production condition learning unit [E ] and a learned condition storage unit [F] that stores the learning results of machine learning in the above-mentioned production condition learning unit [E]. By providing a production condition output section [G] that determines and outputs the adjustment amount of , especially in a plurality of facilities that constitute the metal material production department, the production conditions of each facility can be set by a skilled operator (person) The inventors have found that it is possible to provide a metallic material production system capable of stably producing metallic materials under optimal production conditions at all times without adjustment, and have completed the present invention.

すなわち、本発明の要旨構成は以下のとおりである。
［１］金属材料生産部と機械学習部とを有する金属材料生産システムであって、前記機械学習部は、前記金属材料生産部において実行中の金属材料生産に関する物理量を観測する状態観測部［Ａ］と、前記状態観測部［Ａ］で観測した前記物理量を、データとして記憶する物理量データ記憶部［Ｂ］と、機械学習における報酬条件を設定する報酬条件設定部［Ｃ］と、前記状態観測部［Ａ］で観測した前記物理量のデータ、および前記報酬条件設定部［Ｃ］で設定された前記報酬条件に基づいて報酬を算出する報酬計算部［Ｄ］と、前記報酬計算部［Ｄ］で算出した前記報酬、前記物理量データ、および前記金属材料生産部で設定されている生産条件に基づいて生産条件調整の機械学習を行う生産条件学習部［Ｅ］と、前記生産条件学習部［Ｅ］で機械学習した学習結果を記憶する学習済み条件記憶部［Ｆ］と、前記生産条件学習部［Ｅ］での前記学習結果に基づいて、前記金属材料生産部を構成する各設備の生産条件の調整量を決定して出力する生産条件出力部［Ｇ］と、を備え、前記物理量データ記憶部［Ｂ］、前記報酬条件設定部［Ｃ］、前記報酬計算部［Ｄ］、前記生産条件学習部［Ｅ］および前記学習済み条件記憶部［Ｆ］に、前記金属材料生産部で製造された金属材料を用いて測定した金属材料の表面状態、形状、材料強度および曲げ加工性からなる外部データとして入力し、前記生産条件学習部［Ｅ］の学習に使用することを特徴とする、金属材料生産システム。
［２］前記学習済み記憶部［Ｆ］に記憶された学習結果を前記生産条件学習部［Ｅ］の学習に使用することを特徴とする、［１］に記載の金属材料生産システム。
［３］前記生産条件学習部［Ｅ］で学習した結果を、前記生産条件出力部［Ｇ］に反映させて、前記金属材料生産部の制御ユニットに指示を出すことを特徴とする、［１］または［２］に記載の金属材料生産システム。
［４］前記報酬計算部［Ｄ］は、前記物理量のデータにあらかじめ許容範囲が設定され、前記物理量のデータの数値が、前記許容範囲内に収まると、プラスの報酬を与えるように算出することを特徴とする、［１］～［３］のいずれか１つに記載の金属材料生産システム。
［５］前記報酬計算部［Ｄ］は、前記物理量のデータにあらかじめ許容範囲値が設定され、前記物理量のデータの数値が、前記許容範囲外になると、前記許容範囲の限界値からの前記物理量のデータの数値のずれ幅に応じてマイナスの報酬を与えるように算出することを特徴とする、［１］～［４］のいずれか１つに記載の金属材料生産システム。
［６］金属材料を生産するに際し、状態観測部［Ａ］により、実行中の金属材料生産に関する物理量を観測する工程と、物理量データ記憶部［Ｂ］に前記物理量をデータとして記憶する工程と、報酬条件設定部［Ｃ］により、機械学習における報酬条件を設定する工程と、報酬計算部［Ｄ］により、前記状態観測部［Ａ］で観測した前記物理量のデータ、および前記報酬条件設定部［Ｃ］で設定された前記報酬条件に基づいて報酬を算出する工程と、生産条件学習部［Ｅ］により、前記報酬計算部［Ｄ］が算出した前記報酬、前記物理量データ、および前記金属材料生産部で設定されている生産条件に基づいて生産条件調整の機械学習を行う工程と、学習済み条件記憶部［Ｆ］に前記生産条件学習部［Ｅ］で機械学習した学習結果を記憶する工程と、生産条件出力部［Ｇ］により、前記生産条件学習部［Ｅ］での前記学習結果に基づいて、前記金属材料生産部を構成する各設備の生産条件の調整量を決定して出力する工程と、を備え、前記物理量データ記憶部［Ｂ］、前記報酬条件設定部［Ｃ］、前記報酬計算部［Ｄ］、前記生産条件学習部［Ｅ］および前記学習済み条件記憶部［Ｆ］に、前記金属材料生産部で製造された金属材料を用いて測定した金属材料の表面状態、形状、材料強度および曲げ加工性からなる外部データとして入力し、前記生産条件学習部［Ｅ］の学習に使用することを特徴とする、金属材料生産方法。 That is, the gist and configuration of the present invention are as follows.
[1] A metal material production system having a metal material production unit and a machine learning unit, wherein the machine learning unit is a state observation unit [A ], a physical quantity data storage unit [B] that stores the physical quantity observed by the state observation unit [A] as data, a reward condition setting unit [C] that sets reward conditions in machine learning, and the state observation unit [B] a reward calculation unit [D] for calculating a reward based on the data of the physical quantity observed in the unit [A] and the reward condition set by the reward condition setting unit [C]; and the reward calculation unit [D]. A production condition learning unit [E] that performs machine learning for adjusting production conditions based on the remuneration calculated in , the physical quantity data, and the production conditions set in the metal material production unit; and the production condition learning unit [E ] and a learned condition storage unit [F] that stores the learning results of machine learning in the above-mentioned production condition learning unit [E]. a production condition output unit [G] that determines and outputs the adjustment amount of the physical quantity data storage unit [B], the remuneration condition setting unit [C], the remuneration calculation unit [D], the production conditions In the learning part [E] and the learned condition storage part [F], the surface condition, shape, material strength and bending workability of the metal material measured using the metal material manufactured in the metal material production department are stored. A metal material production system characterized by being input as data and used for learning of the production condition learning section [E] .
[ 2 ] The metal material production system according to [1 ] , characterized in that the learning result stored in the learned storage unit [F] is used for learning of the production condition learning unit [E].
[ 3 ] The result learned by the production condition learning unit [E] is reflected in the production condition output unit [ G], and an instruction is issued to the control unit of the metal material production unit. ] or the metallic material production system according to [2] .
[ 4 ] The remuneration calculation unit [D] has an allowable range set in advance for the data of the physical quantity, and calculates so as to give a positive remuneration when the numerical value of the data of the physical quantity falls within the allowable range. The metal material production system according to any one of [1] to [3] , characterized by:
[ 5 ] The remuneration calculation unit [D] has an allowable range value set in advance for the data of the physical quantity, and when the numerical value of the data of the physical quantity is outside the allowable range, the physical quantity from the limit value of the allowable range The metal material production system according to any one of [1] to [4] , wherein calculation is performed so as to give a negative reward according to the deviation range of the numerical values of the data.
[ 6 ] When producing a metal material, a step of observing a physical quantity related to the production of the metal material being executed by the state observation unit [A], and a step of storing the physical quantity as data in the physical quantity data storage unit [B]; A step of setting a reward condition in machine learning by a reward condition setting unit [C]; A step of calculating a reward based on the reward condition set in step C], and a production condition learning unit [E] for calculating the reward calculated by the reward calculation unit [D], the physical quantity data, and the metal material production a step of performing machine learning of production condition adjustment based on the production conditions set in the production condition learning unit [E]; , a step of determining and outputting the adjustment amount of the production conditions of each facility constituting the metal material production department based on the learning result of the production condition learning department [E] by the production condition output section [G]. and in the physical quantity data storage unit [B], the remuneration condition setting unit [C], the remuneration calculation unit [D], the production condition learning unit [E] and the learned condition storage unit [F] , input as external data consisting of the surface condition, shape, material strength and bending workability of the metal material measured using the metal material manufactured in the metal material production department, and used in the learning of the production condition learning department [E] A metal material production method, characterized by using :

本発明によれば、特に、金属材料生産部を構成する複数の設備において、各設備の生産条件を、熟練オペレータ（人）による調整を行なわなくても、常に最適な生産条件で安定して金属材料を製造することができる。 According to the present invention, in particular, in a plurality of facilities constituting a metal material production department, the production conditions of each facility are always stably produced under optimum production conditions without adjustment by a skilled operator (person). materials can be manufactured.

本実施形態に係る金属材料生産システムの概略模式図である。1 is a schematic diagram of a metallic material production system according to an embodiment; FIG. 本実施形態に係る金属材料生産部の概略模式図である。1 is a schematic diagram of a metal material production department according to the present embodiment; FIG. 機械学習モデルを説明するための概略図である。It is a schematic diagram for explaining a machine learning model.

以下、本発明の実施形態を、図面を参照しながら詳細に説明するが、本発明は以下の実施形態に何ら限定されるものではない。 BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings, but the present invention is not limited to the following embodiments.

＜金属材料生産システム＞
本実施形態の金属材料生産システムは、金属材料生産部と機械学習部とを有する金属材料生産システムであって、前記機械学習部は、前記金属材料生産部において実行中の金属材料生産に関する物理量を観測する状態観測部［Ａ］と、前記状態観測部［Ａ］で観測した前記物理量を、データとして記憶する物理量データ記憶部［Ｂ］と、機械学習における報酬条件を設定する報酬条件設定部［Ｃ］と、前記状態観測部［Ａ］で観測した前記物理量のデータ、および前記報酬条件設定部［Ｃ］で設定された前記報酬条件に基づいて報酬を算出する報酬計算部［Ｄ］と、前記報酬計算部［Ｄ］で算出した前記報酬、前記物理量データ、および前記金属材料生産部で設定されている生産条件に基づいて生産条件調整の機械学習を行う生産条件学習部［Ｅ］と、前記生産条件学習部［Ｅ］で機械学習した学習結果を記憶する学習済み条件記憶部［Ｆ］と、前記生産条件学習部［Ｅ］での前記学習結果に基づいて、前記金属材料生産部を構成する各設備の生産条件の調整量を決定して出力する生産条件出力部［Ｇ］と、を備えることを特徴とするものである。図１は、本実施形態に係る金属材料生産システムの概略模式図である。 <Metal material production system>
A metallic material production system of the present embodiment is a metallic material production system having a metallic material production section and a machine learning section, wherein the machine learning section calculates physical quantities related to metallic material production being executed in the metallic material production section. A state observation unit [A] for observation, a physical quantity data storage unit [B] for storing the physical quantity observed by the state observation unit [A] as data, and a reward condition setting unit [B] for setting reward conditions in machine learning. C], a reward calculation unit [D] that calculates a reward based on the data of the physical quantity observed by the state observation unit [A] and the reward condition set by the reward condition setting unit [C]; a production condition learning unit [E] that performs machine learning for adjusting production conditions based on the remuneration calculated by the remuneration calculation unit [D], the physical quantity data, and the production conditions set in the metal material production unit; A learned condition storage unit [F] for storing learning results obtained by machine learning in the production condition learning unit [E]; and a production condition output unit [G] for determining and outputting the adjustment amount of the production condition of each of the constituent facilities. FIG. 1 is a schematic diagram of a metal material production system according to this embodiment.

このような金属材料生産システムによれば、特に、金属材料生産部を構成する複数の設備において、各設備の生産条件を、熟練オペレータ（人）による調整を行なわなくても、常に最適な生産条件で安定して金属材料を製造することができる。また、条件出し作業が少なくなることから、金属材料の原料のロスが低減され、また、製造される金属材料の物性のばらつきも小さくなるため、コスト面での利点も非常に大きい。 According to such a metal material production system, in particular, in a plurality of facilities constituting the metal material production department, the production conditions of each facility are always optimal production conditions without adjustment by a skilled operator (person). can stably produce metal materials. In addition, since the condition setting work is reduced, the loss of the raw material of the metal material is reduced, and the variation in the physical properties of the manufactured metal material is also reduced, so that there is a great advantage in terms of cost.

〔金属材料生産部〕
金属材料生産部は、金属材料の生産を行うシステムである。このような金属材料生産システムは、例えば溶解・鋳造、均質化熱処理、熱間圧延、表面切削（面削）、冷間圧延、熱処理、表面研磨、防錆処理などを行うことができるそれぞれの工程を有している。そして、これらの装置には、各々多種の条件が存在する。しかも、製造すべき金属材料の機械特性、電気的特性、形状、表面状態などの品質や、材質、幅、板厚などの設計も考慮して、その条件を設計する必要があることから、通常、金属材料の最適な生産条件を決定するには多大な労力やコストを要するが、後述する機械学習部により、各生産工程における最適な生産条件についての機械学習を行うことにより、金属材料の最適な生産条件を決定することができる。 [Metal Material Production Department]
The Metal Material Production Department is a system that produces metal materials. Such a metal material production system can perform, for example, melting/casting, homogenization heat treatment, hot rolling, surface cutting (facing), cold rolling, heat treatment, surface polishing, anti-corrosion treatment, etc. have. Each of these devices has various conditions. In addition, it is necessary to design the conditions by considering the quality of the metal material to be manufactured, such as mechanical properties, electrical properties, shape, and surface condition, as well as design such as material, width, and plate thickness. , It takes a lot of labor and cost to determine the optimum production conditions for metal materials. production conditions can be determined.

具体的に、金属材料生産部において、Ｃｕ－Ｎｉ－Ｓｉ合金の製造を行う場合について図２を用いて説明する。図２は、本実施形態に係る金属材料生産部の概略模式図である。Ｃｕ－Ｎｉ－Ｓｉ合金の生産方法を構成する工程は、まずＣｕに対してＮｉを１．０～５．０質量％、Ｓｉを０．２５～１．２５質量％添加して、溶解鋳造［１］し、その後、保持温度９００℃以上で均質化熱処理［２］を行う。次いで、熱間圧延［３］を行い、鋳塊の約１／１０の板厚に圧延後、水焼き入れ［４］する。次に、表面の酸化膜を除去する面削［５］を行った後、合計の加工率が７０％以上となるよう冷間圧延［６］し、保持温度７００～１０００℃で１秒～６０分の溶体化熱処理［７］を施して急冷を行う。次に、保持温度４００～６００℃で１０分～１２時間の時効析出熱処理［８］を行い、析出強化を行う。その度、表面の酸洗および研磨工程［９］、仕上冷間圧延［１０］、調質焼鈍［１１］の順に行い、０．０５～０．８ｍｍ程度の板厚に仕上げて金属材料製品とする。このようなＣｕ－Ｎｉ－Ｓｉ合金を製造する金属材料生産部は、１１の生産工程が存在する。そして、ほぼ全ての生産工程における生産条件により、生産される金属材料の品質が変化する。 Specifically, a case where a Cu--Ni--Si alloy is produced in the metal material production department will be described with reference to FIG. FIG. 2 is a schematic diagram of the metal material production department according to the present embodiment. The steps constituting the method for producing a Cu—Ni—Si alloy include first adding 1.0 to 5.0% by mass of Ni and 0.25 to 1.25% by mass of Si to Cu, and then melting and casting [ 1], and then a homogenization heat treatment [2] is performed at a holding temperature of 900° C. or higher. Next, hot rolling [3] is performed, and after rolling to a plate thickness of about 1/10 of the ingot, water quenching [4] is performed. Next, after performing chamfering [5] to remove the oxide film on the surface, cold rolling [6] is performed so that the total processing rate is 70% or more, and the holding temperature is 700 to 1000 ° C. for 1 second to 60 minutes. solution heat treatment [7] for 10 minutes and quenching. Next, an aging precipitation heat treatment [8] is performed at a holding temperature of 400 to 600° C. for 10 minutes to 12 hours for precipitation strengthening. Each time, pickling and polishing the surface [9], finishing cold rolling [10], and temper annealing [11] are performed in order to finish the plate thickness of about 0.05 to 0.8 mm, and the metal material product is obtained. do. There are 11 production processes in the metal material production department that manufactures such Cu--Ni--Si alloys. The quality of the produced metal material changes depending on the production conditions in almost all production processes.

本実施形態の金属材料生産システムで製造する金属材料としては、特に限定されず、多種の金属を製造することができる。特に段取り替えが多い生産工程では、段取り替えごとに生産条件の条件出しが必要となるため、本実施形態の金属材料生産システムの導入による、労力やコストの削減効果が大きい。したがって、本実施形態の金属材料生産システムは、少品種大量生産よりも中品種中量生産、ロット生産および多品種少量生産において効果をより発揮し得る。ただし、段取り替えが少ない工程では、オペレータの教育の機会も少なくなるために、生産条件の条件出し作業にかえって大幅な時間を要することが想定される。このような場合などには、本実施形態の金属材料生産システムを導入することで一定の効果が得られる。 The metal material produced by the metal material production system of this embodiment is not particularly limited, and various metals can be produced. Especially in a production process with many setup changes, since it is necessary to determine production conditions for each setup change, the introduction of the metal material production system of the present embodiment has a large labor and cost reduction effect. Therefore, the metal material production system of the present embodiment is more effective in medium-variety medium-volume production, lot production, and high-variety low-volume production than in small-variety mass production. However, in a process in which there are few setup changes, there are fewer opportunities for operator education, so it is assumed that the adjustment of production conditions will take a long time. In such a case, a certain effect can be obtained by introducing the metal material production system of this embodiment.

具体的に、金属材料生産部は、例えば銅合金材料の生産を行うことが好ましい。銅合金材料の製造は、一般に成分系の異なる合金（例えばりん青銅、純銅、Ｃｕ－Ｎｉ－Ｓｉ系合金等）を同じ設備で量産する。多品種少量生産であるため、特定の製品専用の製造ラインを作らずに共通の設備で順番や条件を変更して量産している。したがって、段取り替えが必然的に多くなり、本実施形態の金属材料生産システムの導入による、労力やコストの削減効果が大きい。一方で、例えば、鉄鋼材料の製造は、溶解鋳造から最終の板材に至るまで一貫して連続的に行われる。鉄鋼材料の製造では１つの品種の生産数量が極めて多いために、鋳造機から圧延機などの設備を特定の製品専用の連続製造ライン（単一ライン）として用いて、少品種大量生産を行っている。このような生産では段取り替えは多くない。 Specifically, the metal material production department preferably produces copper alloy materials, for example. Copper alloy materials are generally produced by mass-producing alloys with different component systems (for example, phosphor bronze, pure copper, Cu--Ni--Si alloys, etc.) using the same facility. Because it is a high-mix, low-volume production, we do not create a dedicated production line for a specific product, but mass-produce it by changing the order and conditions with common equipment. Therefore, the number of setup changes inevitably increases, and the introduction of the metal material production system of the present embodiment has a large effect of reducing labor and costs. On the other hand, for example, steel materials are manufactured continuously from melting and casting to the final plate material. In the manufacture of iron and steel materials, the production volume of one product type is extremely large, so facilities such as casting machines and rolling mills are used as a continuous production line (single line) dedicated to a specific product to mass produce a small number of products. there is In such production, there are not many changeovers.

〔機械学習部〕
機械学習部は、金属材料生産部において実行中の金属材料生産に関する物理量を観測する状態観測部［Ａ］と、状態観測部［Ａ］で観測した物理量を、データとして記憶する物理量データ記憶部［Ｂ］と、機械学習における報酬条件を設定する報酬条件設定部［Ｃ］と、状態観測部［Ａ］で観測した物理量のデータ、および報酬条件設定部［Ｃ］に設定された報酬条件に基づいて報酬を算出する報酬計算部［Ｄ］と、報酬計算部［Ｄ］で算出した報酬、物理量データ、および金属材料生産部で設定されている生産条件に基づいて生産条件調整の機械学習を行う生産条件学習部［Ｅ］と、生産条件学習部［Ｅ］で機械学習した学習結果を記憶する学習済み条件記憶部［Ｆ］と、生産条件学習部［Ｅ］での学習結果に基づいて、金属材料生産部を構成する各設備の生産条件の調整量を決定して出力する生産条件出力部［Ｇ］と、を備えるものである。この機械学習部により、上述した金属材料生産部の各設備における生産条件を機械学習して、最適な生産条件を決定することができる。以下、各部の動作について説明する。 [Machine Learning Department]
The machine learning unit consists of a state observation unit [A] that observes physical quantities related to metal material production being executed in the metal material production unit, and a physical quantity data storage unit [ B], the reward condition setting unit [C] that sets the reward conditions in machine learning, the physical quantity data observed by the state observation unit [A], and the reward conditions set in the reward condition setting unit [C] Machine learning for production condition adjustment is performed based on the reward calculation unit [D] that calculates the reward by calculating the reward, the physical quantity data calculated by the reward calculation unit [D], and the production conditions set in the metal material production department. A production condition learning unit [E], a learned condition storage unit [F] that stores learning results obtained by machine learning in the production condition learning unit [E], and based on the learning results in the production condition learning unit [E], and a production condition output unit [G] that determines and outputs the adjustment amount of the production conditions of each facility that constitutes the metal material production unit. The machine learning section can machine-learn the production conditions in each facility of the metal material production section described above to determine the optimum production conditions. The operation of each part will be described below.

（状態観測部［Ａ］）
状態観測部［Ａ］は、金属材料生産部において実行中の金属材料生産に関する物理量を観測するものである。 (State observation part [A])
The state observation section [A] observes physical quantities relating to metal material production being executed in the metal material production section.

状態観測部［Ａ］は、物理量のデータを送信可能な状態で物理量データ記憶部［Ｂ］に接続される。また、状態観測部［Ａ］は、物理量のデータを送信可能な状態で報酬計算部［Ｄ］および生産条件学習部［Ｅ］に接続されてもよい。 The state observation unit [A] is connected to the physical quantity data storage unit [B] in a state in which physical quantity data can be transmitted. Further, the state observation section [A] may be connected to the reward calculation section [D] and the production condition learning section [E] in a state in which physical quantity data can be transmitted.

ここで、物理量は、各設備における処理対象の材料の物性値であればよく、例えば処理対象の材料の形状、寸法、厚さ、表面状態、板材のテンション等が挙げられる。状態観測部［Ａ］においては、これらをセンサー等により測定する。測定したデータは、後述する物理量データ記憶部［Ｂ］に送信されるとともに、報酬計算部［Ｄ］、生産条件学習部［Ｅ］にも送信される。物理量のデータを報酬計算部［Ｄ］、生産条件学習部［Ｅ］に送信する場合において、物理量データ記憶部［Ｂ］を経由してもしなくてもよい。 Here, the physical quantity may be a physical property value of the material to be processed in each facility, and examples thereof include the shape, size, thickness, surface condition, tension of the plate material, etc. of the material to be processed. These are measured by a sensor or the like in the state observation section [A]. The measured data is sent to the physical quantity data storage unit [B], which will be described later, as well as to the remuneration calculation unit [D] and the production condition learning unit [E]. When transmitting the physical quantity data to the remuneration calculation section [D] and the production condition learning section [E], the physical quantity data storage section [B] may or may not be used.

（物理量データ記憶部［Ｂ］）
物理量データ記憶部［Ｂ］は、前記状態観測部［Ａ］で観測した前記物理量を、データとして記憶するものである。物理量データ記憶部は、例えば各種記録媒体（各種メモリ等）であってもよい。 (Physical quantity data storage unit [B])
The physical quantity data storage section [B] stores the physical quantity observed by the state observation section [A] as data. The physical quantity data storage unit may be, for example, various recording media (various memories, etc.).

物理量データ記憶部［Ｂ］は、物理量のデータを受信可能な状態で状態観測部［Ａ］に接続される。また、物理量データ記憶部［Ｂ］は、物理量のデータを送信可能な状態で報酬計算部［Ｄ］および生産条件学習部［Ｅ］に接続されてもよい。 The physical quantity data storage unit [B] is connected to the state observation unit [A] in a state capable of receiving physical quantity data. Also, the physical quantity data storage unit [B] may be connected to the remuneration calculation unit [D] and the production condition learning unit [E] in a state in which physical quantity data can be transmitted.

（報酬条件設定部［Ｃ］）
報酬条件設定部［Ｃ］は、機械学習における報酬条件を設定するものである。この報酬条件設定部［Ｃ］で設定した報酬条件を基に、後述する報酬計算部［Ｄ］で報酬が計算され、その報酬を基に、生産条件学習部［Ｅ］において生産条件を学習して、最適な生産条件を決定する。 (Remuneration condition setting unit [C])
The reward condition setting unit [C] sets reward conditions in machine learning. Based on the reward conditions set by the reward condition setting unit [C], the reward is calculated by the reward calculation unit [D], which will be described later, and the production conditions are learned by the production condition learning unit [E] based on the reward. to determine the optimum production conditions.

報酬条件設定部［Ｃ］は、報酬条件を送信可能な状態で少なくとも報酬計算部［Ｄ］に接続される。 The remuneration condition setting section [C] is connected to at least the remuneration calculation section [D] in a state in which remuneration conditions can be transmitted.

報酬条件としては、例えば、処理対象の材料の形状、寸法、厚さ、表面状態、板材のテンションなど物理量のデータの許容範囲や、それらのばらつきの許容範囲などが挙げられる。なお、報酬条件は、例えば寸法、厚さ、テンションなどの定量的なデータに基づくものであってもよく、また、例えば表面状態の画像データによる良否など定性的なデータに基づくものであってもよい。 Remuneration conditions include, for example, the allowable range of physical quantity data such as the shape, size, thickness, surface condition of the material to be processed, and the tension of the plate material, and the allowable range of their variation. The reward conditions may be based on quantitative data such as dimensions, thickness, and tension, or may be based on qualitative data such as image data of surface conditions. good.

また、報酬条件は、例えば金属加工を行う際に金属材料を用いて測定した金属材料の表面状態、形状、材料強度および曲げ加工性などの外部データに基づいて判断される条件を更に加えてもよい。ここで、「外部データ」とは、当該金属材料生産システムの内部すなわち状態観測部［Ａ］で測定されるような材料の物性とは異なるものであり、その金属材料生産システムで生産し終えた後の金属材料の物性である。 In addition, the reward conditions may further include conditions determined based on external data such as the surface condition, shape, material strength, and bending workability of the metal material measured using the metal material when performing metal processing. good. Here, the "external data" is different from the physical properties of the material as measured inside the metal material production system, that is, in the state observation section [A], and is It is the physical properties of the metal material later.

なお、報酬条件設定部における報酬条件の設定は、作業者のこれまでの経験やこれまで機械学習した結果に基づいて、作業者による手入力又は自動入力で設定を行う。 The setting of remuneration conditions in the remuneration condition setting unit is performed manually or automatically by the worker based on the worker's past experience and machine learning results.

（報酬計算部［Ｄ］）
報酬計算部［Ｄ］は、前記状態観測部［Ａ］で観測した前記物理量のデータ、および前記報酬条件設定部［Ｃ］に設定された前記報酬条件に基づいて報酬を算出するものである。すなわち、報酬計算部［Ｄ］は、前記状態観測部［Ａ］で観測した前記物理量のデータが、前記報酬条件設定部［Ｃ］で設定された前記報酬条件の充足の成否を判断し、または報酬条件の充足・不足の程度を算出し、そしてその報酬条件の充足の成否、または充足・不足の程度に応じてあらかじめ設定された報酬を与えるように算出する。なお、報酬計算部［Ｄ］における報酬の決定方法の詳細は後述する。 (Remuneration Calculation Department [D])
The remuneration calculation unit [D] calculates a remuneration based on the physical quantity data observed by the state observation unit [A] and the remuneration conditions set in the remuneration condition setting unit [C]. That is, the reward calculation unit [D] determines whether the data of the physical quantity observed by the state observation unit [A] satisfies the reward condition set by the reward condition setting unit [C], or The degree of sufficiency/insufficiency of the remuneration conditions is calculated, and a predetermined remuneration is given according to the success or failure of the sufficiency of the remuneration conditions or the degree of sufficiency/insufficiency. The details of the method of determining the remuneration in the remuneration calculation unit [D] will be described later.

報酬計算部［Ｄ］は、物理量のデータを受信可能な状態で状態観測部［Ａ］または物理量データ記憶部［Ｂ］に接続される。また、報酬計算部［Ｄ］は、報酬条件を受信可能な状態で報酬条件設定部［Ｄ］に接続される。さらに、報酬計算部［Ｄ］は、報酬を送信可能な状態で生産条件学習部［Ｅ］に接続される。 The reward calculation unit [D] is connected to the state observation unit [A] or the physical quantity data storage unit [B] in a state capable of receiving physical quantity data. Further, the remuneration calculation unit [D] is connected to the remuneration condition setting unit [D] in a state capable of receiving remuneration conditions. Furthermore, the remuneration calculation section [D] is connected to the production condition learning section [E] in a state in which remuneration can be transmitted.

報酬計算部［Ｄ］は、例えば、前記物理量のデータにあらかじめ許容範囲が設定され、前記物理量のデータの数値が、前記許容範囲内に収まると、プラスの報酬を与えるように算出し、一方で、前記物理量のデータの数値が、前記許容範囲外になると、前記許容範囲の限界値からの前記物理量のデータの数値のずれ幅に応じてマイナスの報酬を与えるように算出することができる。 The reward calculation unit [D], for example, has an allowable range set in advance for the data of the physical quantity, and calculates so as to give a positive reward when the numerical value of the data of the physical quantity falls within the allowable range, , when the numerical value of the data of the physical quantity falls outside the allowable range, a negative reward is given according to the deviation width of the numerical value of the data of the physical quantity from the limit value of the allowable range.

報酬計算部［Ｄ］は、例えば、良好な表面状態の画像データ、厚さのばらつきが小さいデータおよび板材のテンションのばらつきが小さいデータのうちの少なくとも１つのデータを、その寄与の程度に応じてプラスの報酬を与えるように算出する。一方で、報酬計算部［Ｄ］が、表面状態の粗い画像データ、厚さのばらつきが大きいデータおよび板材のテンションのばらつきが大きいデータのうちの少なくとも１つのデータを、その寄与の程度に応じてマイナスの報酬を与えるように算出する。 The reward calculation unit [D], for example, calculates at least one of image data of good surface condition, data with small variation in thickness, and data with small variation in tension of the plate according to the degree of contribution. Calculate to give a positive reward. On the other hand, the reward calculation unit [D] calculates at least one data from among image data with rough surface conditions, data with large variation in thickness, and data with large variation in tension of the plate material, according to the degree of contribution. Calculated to give a negative reward.

（生産条件学習部［Ｅ］）
生産条件学習部［Ｅ］は、報酬計算部［Ｄ］で算出した報酬、物理量データ、および金属材料生産部で設定されている生産条件に基づいて生産条件調整の機械学習を行うものである。このようにして、生産条件学習部［Ｅ］は、報酬計算部［Ｄ］で重みづけした報酬の計算結果と、物理量データ記憶部で記憶したデータを基に、その金属材料を生産したときの生産条件によって機械学習を行う。 (Production condition learning department [E])
The production condition learning unit [E] performs machine learning for production condition adjustment based on the reward calculated by the reward calculation unit [D], the physical quantity data, and the production conditions set by the metal material production department. In this way, the production condition learning unit [E] uses the calculation results of the weighted rewards in the reward calculation unit [D] and the data stored in the physical quantity data storage unit to produce the metal material. Perform machine learning according to production conditions.

生産条件学習部［Ｅ］は、物理量のデータを受信可能な状態で状態観測部［Ａ］または物理量データ記憶部［Ｂ］に接続される。また、生産条件学習部［Ｅ］は、報酬を受信可能な状態で報酬計算部［Ｄ］に接続される。さらに、生産条件学習部［Ｅ］は、学習結果を送信可能な状態で学習済み条件記憶部［Ｆ］および生産条件出力部［Ｇ］に接続される。なお、生産条件学習部［Ｅ］は、学習結果を送信可能な状態で学習済み条件記憶部［Ｆ］に受信可能な状態で報酬計算部［Ｄ］に接続されてもよい。 The production condition learning section [E] is connected to the state observation section [A] or the physical quantity data storage section [B] in a state capable of receiving physical quantity data. Also, the production condition learning unit [E] is connected to the reward calculation unit [D] in a state where it can receive rewards. Furthermore, the production condition learning unit [E] is connected to the learned condition storage unit [F] and the production condition output unit [G] in a state in which learning results can be transmitted. The production condition learning unit [E] may be connected to the remuneration calculation unit [D] in a state in which learning results can be transmitted to the learned condition storage unit [F] in a state in which they can be received.

前記物理量データ記憶部［Ｂ］、前記報酬条件設定部［Ｃ］、前記報酬計算部［Ｄ］、前記生産条件学習部［Ｅ］および前記学習済み条件記憶部［Ｆ］の少なくともいずれか１つに、前記金属材料生産部で製造された金属材料を用いて測定した金属材料の表面状態、形状、材料強度および曲げ加工性からなる外部データとして入力し、前記生産条件学習部［Ｅ］の学習に使用することもできる。 At least one of the physical quantity data storage unit [B], the remuneration condition setting unit [C], the remuneration calculation unit [D], the production condition learning unit [E], and the learned condition storage unit [F] , input as external data consisting of the surface condition, shape, material strength and bending workability of the metal material measured using the metal material manufactured in the metal material production department, and the learning of the production condition learning department [E] can also be used for

また、下記で説明する学習済み記憶部［Ｆ］に記憶された学習結果を前記生産条件学習部［Ｅ］の学習に使用することができる。 Further, the learning results stored in the learned memory section [F], which will be described below, can be used for learning by the production condition learning section [E].

なお、金属材料の生産時において、状態観測部［Ａ］の物理量の観測にともなう生産条件の更新は、逐次に行ってもよく、また、一定の時期に行ってもよい。 During the production of the metal material, the production conditions may be updated in accordance with the observation of the physical quantity by the state observation unit [A], either sequentially or at a certain time.

（学習済み条件記憶部［Ｆ］）
学習済み条件記憶部［Ｆ］は、前記生産条件学習部［Ｅ］で機械学習した学習結果を記憶するものである。また、好ましくは、前記生産条件学習部［Ｅ］に学習結果を送信して、反映させることができるものである。学習済み条件記憶部［Ｆ］は、例えば各種記録媒体（各種メモリ等）であってよい。また、学習済み条件記憶部［Ｆ］は、物理量データ記憶部［Ｂ］と同一のものであってもよい。 (Learned condition storage unit [F])
The learned condition storage unit [F] stores the results of machine learning performed by the production condition learning unit [E]. Moreover, preferably, the learning result can be transmitted to the production condition learning section [E] and reflected. The learned condition storage unit [F] may be, for example, various recording media (various memories, etc.). Also, the learned condition storage unit [F] may be the same as the physical quantity data storage unit [B].

学習済み条件記憶部［Ｆ］は、学習結果を受信可能な状態で少なくとも生産条件学習部［Ｅ］に接続される。また、学習済み条件記憶部［Ｆ］は、学習結果を送信可能な状態で少なくとも生産条件学習部［Ｅ］に接続されてもよい。 The learned condition storage unit [F] is connected to at least the production condition learning unit [E] in a state where learning results can be received. Also, the learned condition storage section [F] may be connected to at least the production condition learning section [E] in a state in which learning results can be transmitted.

学習済み条件記憶部において記憶するデータは、生産された金属材料の報酬の計算結果と、物理量データ記憶部と、その金属材料を生産したときに金属材料生産部に設定されている生産条件の対応関係であってよい。 The data to be stored in the learned condition storage section are the calculation result of the reward for the produced metal material, the physical quantity data storage section, and the correspondence between the production conditions set in the metal material production section when the metal material was produced. It can be a relationship.

（生産条件出力部［Ｇ］）
生産条件出力部［Ｇ］は、前記生産条件学習部［Ｅ］での前記学習結果に基づいて、前記金属材料生産部で製造される金属材料の生産条件の調整量を決定して出力するものである。 (Production condition output section [G])
The production condition output unit [G] determines and outputs the adjustment amount of the production condition of the metal material manufactured by the metal material production unit based on the learning result of the production condition learning unit [E]. is.

生産条件出力部［Ｇ］は、学習結果を受信可能な状態で少なくとも生産条件学習部［Ｅ］に接続される。また、生産条件出力部［Ｇ］は、生産条件の調整量を送信可能な状態で金属材料生産部の各工程の設備に接続される。 The production condition output section [G] is connected to at least the production condition learning section [E] in a state capable of receiving learning results. In addition, the production condition output unit [G] is connected to the equipment of each process of the metal material production unit in a state in which adjustment amounts of production conditions can be transmitted.

この生産条件出力部［Ｇ］は、金属材料生産部の制御ユニットと通信可能な状態で接続するなどして、前記金属材料生産部の制御ユニットに指示を出すように構成してもよい。このような構成とすることで、金属材料生産システムの自動化を達成することができる。 This production condition output unit [G] may be configured to issue instructions to the control unit of the metal material production department by connecting it in a communicable state with the control unit of the metal material production department. With such a configuration, automation of the metal material production system can be achieved.

なお、機械学習部の状態観測部［Ａ］、物理量データ記憶部［Ｂ］、報酬条件設定部［Ｃ］、報酬計算部［Ｄ］、生産条件学習部［Ｅ］、学習済み条件記憶部［Ｆ］および生産条件出力部［Ｇ］は、各々が通信可能な状態で接続されていてよい。 The state observation unit [A] of the machine learning unit, the physical quantity data storage unit [B], the reward condition setting unit [C], the reward calculation unit [D], the production condition learning unit [E], the learned condition storage unit [ F] and the production condition output unit [G] may be connected in a communicable state.

〔機械学習〕
以下、本実施形態の金属材料生産システムにおける報酬条件設定部［Ｃ］、報酬計算部［Ｄ］および生産条件学習部［Ｅ］で行う機械学習について、状態価値関数と行動価値関数を使用して説明する。 [Machine learning]
Machine learning performed by the reward condition setting unit [C], the reward calculation unit [D], and the production condition learning unit [E] in the metal material production system of this embodiment will be described below using the state value function and the action value function. explain.

強化学習では、環境から得られる最終的な累積報酬を最大化することで学習を行う。累積報酬は下記の式で与えられる。図３は、機械学習モデルを説明するための概略図である。

Figure 0007189777000001
（ここで、Ｔは最終時刻、γは遠い将来に得られる報酬ほど割り引いて評価するための割引率であり、０≦γ≦１である。） Reinforcement learning learns by maximizing the final cumulative reward obtained from the environment. The cumulative reward is given by the formula below. FIG. 3 is a schematic diagram for explaining a machine learning model.
Figure 0007189777000001
 (Here, T is the final time, γ is a discount rate for discounting and evaluating rewards obtained in the distant future, and 0 ≤ γ ≤ 1.)

強化学習では、報酬を評価してその評価を最大化することで学習を行う。ここでは、現在の状態がどのくらい良いのかを測る関数として価値関数を考える。どのくらい良いのか、ということを、将来にわたって得られる報酬によって定義する。 Reinforcement learning learns by evaluating rewards and maximizing the evaluation. Here we consider the value function as a function that measures how good the current state is. How good is defined by future rewards.

方策πは、状態ｓ∈Ｓで行動ａ∈Ａ（ｓ）をとることであり、π（ｓ，a）と表す。
方策πのもとで、状態ｓの価値は下記の状態価値関数（ｓｔａｔｅ ｖａｌｕｅ ｆｕｎｃｔｉｏｎ ｆｏｒ ｐｏｌｉｃｙ π）で定式化できる。

Figure 0007189777000002
A policy π is to take an action aεA(s) in a state sεS, denoted π(s,a).
Under policy π, the value of state s can be formulated by the following state value function for policy π.
Figure 0007189777000002

状態価値関数Ｖπ（ｓ）は、ある状態ｓがどのくらい良い状態であるのかを示す価値関数である。状態価値関数は、状態を引数とする関数として表現され、行動を繰り返す中での学習において、ある状態における行動に対して得られた報酬や、該行動により移行する未来の状態の価値などに基づいて更新される。 A state value function V π (s) is a value function that indicates how good a certain state s is. The state-value function is expressed as a function with a state as an argument, and is based on the reward obtained for the action in a certain state in learning while repeating the action, the value of the future state that the action transitions to, etc. updated.

方策πのもとで、状態ｓにおいて行動ａをとることの価値は、下記の行動価値関数（ａｃｔｉｏｎ ｖａｌｕｅ ｆｕｎｃｔｉｏｎ ｆｏｒ ｐｏｌｉｃｙ π）によって定義できる。

Figure 0007189777000003
Under policy π, the value of taking action a in state s can be defined by the following action value function for policy π.
Figure 0007189777000003

行動価値関数Ｑπ（ｓ，ａ）は、ある状態ｓにおいて行動ａがどのくらい良い行動であるのかを示す価値関数である。行動価値関数は、状態と行動を引数とする関数として表現され、行動を繰り返す中での学習において、ある状態における行動に対して得られた報酬や、該行動により移行する未来の状態における行動の価値などに基づいて更新される。 The action value function Q π (s, a) is a value function that indicates how good action a is in a certain state s. The action-value function is expressed as a function with state and action as arguments. In learning during repeated actions, the reward obtained for action in a certain state and the action value in the future state to which the action transitions. Updated based on value, etc.

個の価値関数を記憶する方法としては、近似関数を用いる方法や、配列を用いる方法以外にも、例えば状態ｓが多くの状態をとるような場合には状態ｓｔ、行動ａｔを入力として価値(評価)を出力する多値出力のサポートベクターマシン（ＳＶＭ）やニューラルネットワークなどの教師あり学習器を用いるようにしてもよい。 In addition to the method of using an approximate function and the method of using an array, for example, when the state s takes many states, the state s t and the action a t are used as inputs. A supervised learning device such as a multi-value output support vector machine (SVM) or a neural network that outputs values (evaluations) may be used.

機械学習は、以下の（１）～（５）の繰り返しによって進められる。
（１）環境の状態ｓ（ｓを観測）
（２）行動ａ（観測結果と過去の学習に基づいて自分が取れる行動ａｔを選択して行動ａｔを実行）
（３）環境の状態ｓの変化（行動ａｔが実行されることで、環境の状態ｓｔが次の状態ｓｔ＋１へ変化）
（４）報酬ｒ（行動ａｔの結果としての状態変化に基づいて、機械学習器が報酬ｒｔ＋１を受け取る）
（５）報酬ｒに基づく学習（エージェントが状態ｓｔ、行動ａｔ、報酬ｒｔ＋１および過去の学習結果に基づいて学習を進める。） Machine learning proceeds by repeating the following (1) to (5).
(1) Environmental state s (observing s)
(2) Behavior a (choose action a t that you can take based on observation results and past learning, and execute action a t )
(3) Change in environmental state s (execution of action at causes environmental state s t to change to next state s t +1 )
(4) reward r (the machine learner receives a reward r t +1 based on the state change resulting from the action at)
(5) Learning based on reward r (Agent advances learning based on state s t , action a t , reward r t+1 and past learning results.)

ある環境において、学習が終了した後に、新たな環境に置かれた場合でも追加の学習を行うことでその環境に適応するように学習を進めることができる。よって、本発明のように生産設備（金属材料生産部）の最適条件の算出に適応することで、これまでにないような金属材料の厚さ、長さ、幅、硬さ、表面状態などから最適な加工条件を条件出しなどによって決める必要がなくなり、大幅な時間短縮が可能となる。 Even if you are placed in a new environment after completing the learning in a certain environment, you can proceed with the learning so as to adapt to the new environment by performing additional learning. Therefore, by adapting to the calculation of the optimum conditions for the production facility (metal material production department) like the present invention, it is possible to calculate the thickness, length, width, hardness, surface condition, etc. of the metal material like never before. It eliminates the need to decide the optimum machining conditions by setting conditions, etc., and it is possible to significantly reduce the time.

そして、以上のような金属材料生産システムにおいては、金属材料の生産時には、金属材料生産部の各工程で測定した物理量データを観測する状態観測部［Ａ］によって機械学習を行う。また、生産終了後出荷前の最終製品の引張試験の結果や表面状態からなる外部データや、出荷後の金属材料加工後の金属材料の表面状態、形状、材料強度（引張試験の結果など）および曲げ加工性からなる外部データを用いて機械学習を行う。このようにしてより多くのデータを用いることで、設備履歴と性能、表面状態などのパラメータが追加され、より最適な加工条件を算出することができ、製造時間とオフゲージを大幅に低減することができる。なお、このような場合において、物理量データや外部データは一度物理量データ記憶部［Ｂ］に保存してもよい。また、外部データは逐次にインターネットを経由して金属材料生産システムの各部で授受してもよい。 In the metal material production system as described above, machine learning is performed by the state observation unit [A] that observes the physical quantity data measured in each process of the metal material production unit during the production of the metal material. In addition, external data consisting of the results of tensile tests and surface conditions of final products before shipment after the end of production, surface conditions, shapes, material strength (results of tensile tests, etc.) and Machine learning is performed using external data consisting of bendability. By using more data in this way, additional parameters such as equipment history and performance, surface condition, etc., can be calculated, making it possible to calculate more optimal processing conditions, which can significantly reduce manufacturing time and off-gauge. can. In such a case, the physical quantity data and the external data may be once stored in the physical quantity data storage unit [B]. Also, the external data may be sequentially transferred to and received from each part of the metal material production system via the Internet.

〔金属材料生産システムの動作の具体例〕
上述したＣｕ－Ｎｉ－Ｓｉ合金の工程を一例として、本実施形態の金属材料生産システムの動作をより具体的に説明する。 [Specific example of operation of metal material production system]
The operation of the metal material production system according to the present embodiment will be described more specifically, taking the aforementioned Cu--Ni--Si alloy process as an example.

金属材料生産システムの導入前において、工場の作業日誌または設備の端末に記録したデータ、すなわち入力溶解鋳造［１］の溶解温度、均質化熱処理［２］の測定温度および保持時間、熱間圧延［３］の圧延速度および各圧延パスの圧延加工率、水焼き入れ［４］時の水の流量、面削［５］の回数および面削寸法、冷間圧延［６］の圧延加工率、圧延パス数および圧延速度、溶体化熱処理［７］の昇温速度、到達温度および冷却速度、時効析出熱処理［８］の到達温度、保持時間および冷却速度、研磨工程［９］の研磨速度および研磨紙の番手、仕上冷間圧延［１０］の圧延加工率、圧延パス数および圧延速度、ならびに調質焼鈍［１１］の昇温速度および到達温度について、機械学習させる。生産条件学習済み記憶部から生産条件をオンラインまたはオフラインのいずれかで各設備に指示し、条件をセットし、金属材料生産を行う。 Before the introduction of the metal material production system, the data recorded in the work log of the factory or the terminal of the equipment, that is, the melting temperature of the input melting and casting [1], the measured temperature and holding time of the homogenization heat treatment [2], hot rolling [ 3] rolling speed and rolling rate of each rolling pass, water flow rate during water quenching [4], number of times of facing [5] and facing dimensions, rolling rate of cold rolling [6], rolling Number of passes and rolling speed, heating rate, ultimate temperature and cooling rate of solution heat treatment [7], ultimate temperature, holding time and cooling rate of aging precipitation heat treatment [8], polishing rate and abrasive paper of polishing step [9] Machine learning is performed for the grain size, the rolling reduction rate of the finish cold rolling [10], the number of rolling passes and the rolling speed, and the temperature increase rate and the ultimate temperature of the temper annealing [11]. The production conditions are instructed to each facility either online or offline from the production condition learned storage unit, the conditions are set, and metal material production is performed.

このようにして金属材料生産を行なっている際に、例えば冷間圧延［６］における圧延後の形状や板厚（物理量）が不均一となった場合に、この変化を複数のセンサー（状態観測部［Ａ］）で読み取り、報酬計算部［Ｄ］および生産条件学習部［Ｅ］に伝達する。報酬計算部［Ｄ］では、圧延後の形状や板厚の不均一性に対しマイナスの報酬を与えるように算出し、生産条件学習部［Ｅ］に送信する。生産条件学習部［Ｅ］では、報酬、圧延後の形状や板厚の不均一性および冷間圧延［６］を行ったときに設定されている圧延条件（圧延機やその他設備の設定）に基づいて機械学習を行い、最適化する。より具体的には、冷間圧延では、板材の幅方向の張力や幅方向での圧延ロールのギャップなどを微調整することで、最適な形状の金属材料を得ることができる。主に圧延速度や圧延ロールの幅方向でのギャップ制御を最適化する。そしてこの最適化後の生産条件を、圧延機やその他設備にフィードバックし、次の製造ロットから条件を最適化する。ここで、生産条件学習部［Ｅ］においては、金属材料の表面状態、形状、材料強度、曲げ加工性からなる外部データをさらに用いてもよい。 During the production of metal materials in this way, for example, if the shape or thickness (physical quantity) after rolling in cold rolling [6] becomes uneven, this change is detected by multiple sensors (state observation It is read by the section [A]) and transmitted to the remuneration calculation section [D] and the production condition learning section [E]. The reward calculation unit [D] calculates so as to give a negative reward for non-uniformity in the shape and thickness after rolling, and transmits it to the production condition learning unit [E]. In the production condition learning part [E], the rolling conditions (settings of rolling mills and other equipment) set at the time of remuneration, non-uniformity of shape and thickness after rolling, and cold rolling [6] Machine learning and optimization based on More specifically, in cold rolling, a metal material having an optimum shape can be obtained by finely adjusting the tension in the width direction of the plate material, the gap of the rolls in the width direction, and the like. It mainly optimizes the rolling speed and the gap control in the width direction of the rolling rolls. Then, the production conditions after this optimization are fed back to the rolling mill and other equipment, and the conditions are optimized from the next production lot. Here, in the production condition learning unit [E], external data including the surface condition, shape, material strength, and bending workability of the metal material may be used.

次に、同一の金属材料生産システムで、Ｃｕ－Ｎｉ－Ｓｉ合金とは異なる組成の銅合金を生産する場合について説明する。Ｃｕ－Ｎｉ－Ｓｉ合金を例に説明したことと同様に、工場の作業日誌または設備の端末に記録したデータ（検査成績のデータ）を学習する。このような学習を複数種の組成・添加元素の合金で繰り返すことにより、各工程の最適な生産条件を機械学習するとともに、それぞれの合金の組成・添加元素の分量をその差異的な生産条件と関連付け、その結果を学習済み条件記憶部［Ｆ］に記憶することができる。そして、生産開始後、合金の組成および添加元素の分量を報酬条件設定部［Ｃ］に入力し、その合金の組成および添加元素の分量のデータより、学習済み条件記憶部［Ｆ］に最も近い学習済み生産条件を呼び出す。学習済み条件記憶部から生産条件をオフラインまたはオンラインのいずれかで各設備に指示し、生産条件をセットし、金属材料生産を行う。 Next, a case of producing a copper alloy having a composition different from that of a Cu--Ni--Si alloy using the same metal material production system will be described. In the same way as explained with the Cu--Ni--Si alloy as an example, data (inspection result data) recorded in the work diary of the factory or the terminal of the equipment is learned. By repeating this kind of learning with multiple types of alloys with different compositions and additive elements, we can machine-learn the optimal production conditions for each process, as well as determine the composition and amount of additive elements for each alloy as their different production conditions. association, and the result can be stored in the learned condition storage unit [F]. Then, after the start of production, input the alloy composition and the amount of the additive element into the reward condition setting section [C], and from the data of the alloy composition and the amount of the additive element, the closest to the learned condition storage section [F] Call learned production conditions. The production conditions are instructed to each facility either off-line or on-line from the learned condition storage unit, the production conditions are set, and metal materials are produced.

＜金属材料生産方法＞
本実施形態の金属材料生産方法は、金属材料を生産するに際し、状態観測部［Ａ］により、実行中の金属材料生産に関する物理量を観測する工程と、物理量データ記憶部［Ｂ］に物理量をデータとして記憶する工程と、報酬条件設定部［Ｃ］により、機械学習における報酬条件を設定する工程と、報酬計算部［Ｄ］により、状態観測部［Ａ］で観測した物理量のデータ、および報酬条件設定部［Ｃ］に設定された報酬条件に基づいて報酬を算出する工程と、生産条件学習部［Ｅ］により、報酬計算部［Ｄ］が算出した報酬、物理量データ、および金属材料生産部に設定されている生産条件に基づいて生産条件調整の機械学習を行う工程と、学習済み条件記憶部［Ｆ］に生産条件学習部［Ｅ］で機械学習した学習結果を記憶する工程と、生産条件出力部［Ｇ］により、生産条件学習部［Ｅ］での学習結果に基づいて、金属材料生産部を構成する各設備の生産条件の調整量を決定して出力する工程と、を備えることを特徴とするものである。 <Metal material production method>
The metal material production method of the present embodiment includes, when producing a metal material, the state observation unit [A] observes the physical quantity related to the metal material production being executed, and the physical quantity data storage unit [B] stores the physical quantity as data. a step of setting a reward condition in machine learning by a reward condition setting unit [C]; A step of calculating a reward based on the reward conditions set in the setting unit [C]; A step of performing machine learning for adjusting production conditions based on the set production conditions, a step of storing the learning result of the machine learning in the production condition learning unit [E] in the learned condition storage unit [F], and a production condition and an output unit [G], based on the learning result of the production condition learning unit [E], determines and outputs the adjustment amount of the production conditions of each facility constituting the metal material production unit. It is characterized.

すなわち、このような金属材料生産方法によれば、上記の金属材料生産システムにより、金属材料の生産工程において、金属材料の最適な生産条件を決定することができ、人間による生産条件の調整の手間を抑制することができる。また、条件出し作業が少なくなることから、金属材料の原料のロスが低減され、また、製造される金属材料の物性のばらつきも小さくなるため、コスト面での利点も非常に大きい。 That is, according to such a metal material production method, the above-described metal material production system can determine the optimum production conditions for the metal material in the production process of the metal material, eliminating the need for humans to adjust the production conditions. can be suppressed. In addition, since the condition setting work is reduced, the loss of the raw material of the metal material is reduced, and the variation in the physical properties of the manufactured metal material is also reduced, so that there is a great advantage in terms of cost.

次に、本発明の効果をさらに明確にするために、本発明例について説明するが、本発明はこれら実施例に限定されるものではない。 Next, examples of the present invention will be described in order to further clarify the effects of the present invention, but the present invention is not limited to these examples.

金属材料メーカーＡ社では、図２に示す金属材料生産部を用いてＣｕ－Ｎｉ－Ｓｉ合金を断続的に製造している。Ｃｕ－Ｎｉ－Ｓｉ合金の生産工程は、まずＣｕに対してＮｉを１．０～５．０質量％、Ｓｉを０．２５～１．２５質量％添加して、溶解鋳造［１］し、その後、保持温度９００℃以上で均質化熱処理［２］を行う。次いで、熱間圧延［３］を行い、鋳塊の約１／１０の板厚に圧延後、水焼き入れ［４］する。次に、表面の酸化膜を除去する面削［５］を行った後、合計の加工率が７０％以上となるよう冷間圧延［６］し、保持温度７００～１０００℃で１秒～６０分の溶体化熱処理［７］を施して急冷を行う。次に、保持温度４００～６００℃で１０分～１２時間の時効析出熱処理［８］を行い、析出強化を行う。その度、表面の酸洗および研磨工程［９］、仕上冷間圧延［１０］、調質焼鈍［１１］の順に行い、０．０５～０．８ｍｍ程度の板厚に仕上げて金属材料製品とする。 A metal material manufacturer A uses the metal material production department shown in FIG. 2 to intermittently manufacture a Cu--Ni--Si alloy. The production process of the Cu-Ni-Si alloy includes first adding 1.0 to 5.0% by mass of Ni and 0.25 to 1.25% by mass of Si to Cu, melting and casting [1], After that, a homogenization heat treatment [2] is performed at a holding temperature of 900° C. or higher. Next, hot rolling [3] is performed, and after rolling to a plate thickness of about 1/10 of the ingot, water quenching [4] is performed. Next, after performing chamfering [5] to remove the oxide film on the surface, cold rolling [6] is performed so that the total processing rate is 70% or more, and the holding temperature is 700 to 1000 ° C. for 1 second to 60 minutes. solution heat treatment [7] for 10 minutes and quenching. Next, an aging precipitation heat treatment [8] is performed at a holding temperature of 400 to 600° C. for 10 minutes to 12 hours for precipitation strengthening. Each time, pickling and polishing the surface [9], finishing cold rolling [10], and temper annealing [11] are performed in order to finish the plate thickness of about 0.05 to 0.8 mm, and the metal material product is obtained. do.

Ａ社は、このＣｕ－Ｎｉ－Ｓｉ合金以外にも同一の設備で複数のＣｕ系合金を製造しており、Ｃｕ－Ｎｉ－Ｓｉ合金の製造開始にあたっては、都度、製造工程の条件出し作業を行っている。 In addition to this Cu-Ni-Si alloy, Company A manufactures multiple Cu-based alloys with the same equipment, and at the start of production of Cu-Ni-Si alloys, each time the manufacturing process conditions are determined. Is going.

本発明の金属材料生産システムの導入前、Ａ社では、製造工程の条件出し作業として、各条件についてオペレータが確認を行っていた。この場合における、Ｃｕ－Ｎｉ－Ｓｉ合金の製造開始から出荷までの合計時間は平均して８日間であった。また、各設備での条件出し時に発生する材料の端部の材料ロス（オフゲージ）は５％であった。さらに、オペレータにより生産条件に大きな差異が生じていた。 Before the introduction of the metallic material production system of the present invention, in company A, the operator confirmed each condition as condition setting work for the manufacturing process. In this case, the total time from the start of production of the Cu--Ni--Si alloy to the shipment was 8 days on average. In addition, the material loss (off-gauge) at the end of the material that occurred during conditioning in each facility was 5%. Furthermore, there were large differences in production conditions depending on the operator.

その後、金属材料メーカーＡ社では、本発明の金属材料生産システムを導入した。この金属材料生産システムでは、外部データを使用せずに、これまでの工場の作業日誌または設備の端末に記録したデータ、すなわち入力溶解鋳造［１］の溶解温度、均質化熱処理［２］の測定温度および保持時間、熱間圧延［３］の圧延速度および各圧延パスの圧延加工率、水焼き入れ［４］時の水の流量、面削［５］の回数および面削寸法、冷間圧延［６］の圧延加工率、圧延パス数および圧延速度、溶体化熱処理［７］の昇温速度、到達温度および冷却速度、時効析出熱処理［８］の到達温度、保持時間および冷却速度、研磨工程［９］の研磨速度および研磨紙の番手、仕上冷間圧延［１０］の圧延加工率、圧延パス数および圧延速度、ならびに調質焼鈍［１１］の昇温速度および到達温度について、機械学習させた。生産条件学習済み記憶部から生産条件をオンラインで各設備に指示し、条件をセットし、金属材料生産を行った。 After that, the metallic material manufacturer A introduced the metallic material production system of the present invention. In this metal material production system, without using external data, the data recorded in the work diary of the factory or the terminal of the equipment, that is, the melting temperature of the input melting and casting [1], the measurement of the homogenization heat treatment [2] Temperature and holding time, rolling speed of hot rolling [3] and rolling reduction rate of each rolling pass, flow rate of water during water quenching [4], number and size of facing [5], cold rolling Rolling rate, number of rolling passes and rolling speed of [6], heating rate, ultimate temperature and cooling rate of solution heat treatment [7], ultimate temperature, holding time and cooling rate of aging precipitation heat treatment [8], polishing process The polishing rate and abrasive paper count of [9], the rolling reduction rate, number of rolling passes and rolling speed of finish cold rolling [10], and the heating rate and ultimate temperature of temper annealing [11] were machine-learned. rice field. The production conditions were instructed to each facility online from the production condition learned storage unit, the conditions were set, and metal materials were produced.

この結果、製造工程の条件出し作業としての、溶体化熱処理［７］や冷間圧延［６］の条件についてオペレータによる目視での確認作業や台帳を見て過去の条件との詳細な確認作業、手作業による圧延ロールギャップや焼鈍速度の調整の条件についての検出器による確認作業が短縮され、Ｃｕ－Ｎｉ－Ｓｉ合金の製造開始から出荷までの合計時間は平均して７日間に短縮された。また、各設備での条件出し時に発生する材料の端部の材料ロス（オフゲージ）が４％に減少した。さらに、オペレータにより生産条件に大きな差異が生じなかった。 As a result, the operator visually confirmed the conditions of solution heat treatment [7] and cold rolling [6] as conditions for the manufacturing process, and detailed confirmation work with past conditions by looking at the ledger. The work of manually checking the conditions for adjusting the rolling roll gap and annealing speed using a detector was shortened, and the total time from the start of production of the Cu—Ni—Si alloy to shipment was shortened to 7 days on average. In addition, the material loss (off-gauge) at the end of the material that occurred during condition setting in each facility was reduced to 4%. Furthermore, there were no large differences in production conditions between operators.

その後、金属材料メーカーＡ社では、さらに金属材料の表面、金属材料の形状、材料強度および曲げ加工性のデータを外部データとして用いて、金属材料の生産を行なった。この結果、製造工程の条件出し作業としての、Ｃｕ－Ｎｉ－Ｓｉ合金の製造開始から出荷までの合計時間は平均して４日間に短縮された。また、各設備での条件出し時に発生する材料の端部の材料ロス（オフゲージ）が１％に減少した。さらに、オペレータにより生産条件に大きな差異が生じなかった。 After that, the metal material manufacturer A manufactured metal materials using the surface of the metal material, the shape of the metal material, the strength of the material, and the bending workability as external data. As a result, the total time from the start of the production of the Cu--Ni--Si alloy to the shipment was reduced to 4 days on average, which is the condition setting work for the production process. In addition, the material loss (off-gauge) at the end of the material that occurs during condition setting in each facility has been reduced to 1%. Furthermore, there were no large differences in production conditions between operators.

Claims (6)

金属材料生産部と機械学習部とを有する金属材料生産システムであって、
前記機械学習部は、
前記金属材料生産部において実行中の金属材料生産に関する物理量を観測する状態観測部［Ａ］と、
前記状態観測部［Ａ］で観測した前記物理量を、データとして記憶する物理量データ記憶部［Ｂ］と、
機械学習における報酬条件を設定する報酬条件設定部［Ｃ］と、
前記状態観測部［Ａ］で観測した前記物理量のデータ、および前記報酬条件設定部［Ｃ］で設定された前記報酬条件に基づいて報酬を算出する報酬計算部［Ｄ］と、
前記報酬計算部［Ｄ］で算出した前記報酬、前記物理量データ、および前記金属材料生産部で設定されている生産条件に基づいて生産条件調整の機械学習を行う生産条件学習部［Ｅ］と、
前記生産条件学習部［Ｅ］で機械学習した学習結果を記憶する学習済み条件記憶部［Ｆ］と、
前記生産条件学習部［Ｅ］での前記学習結果に基づいて、前記金属材料生産部を構成する各設備の生産条件の調整量を決定して出力する生産条件出力部［Ｇ］と、を備え、
前記物理量データ記憶部［Ｂ］、前記報酬条件設定部［Ｃ］、前記報酬計算部［Ｄ］、前記生産条件学習部［Ｅ］および前記学習済み条件記憶部［Ｆ］の少なくともいずれか１つに、前記金属材料生産部で製造された金属材料を用いて測定した金属材料の表面状態、形状、材料強度および曲げ加工性からなる外部データとして入力し、前記生産条件学習部［Ｅ］の学習に使用することを特徴とする、金属材料生産システム。 A metal material production system having a metal material production unit and a machine learning unit,
The machine learning unit
a state observation unit [A] that observes physical quantities related to metal material production being executed in the metal material production unit;
a physical quantity data storage unit [B] that stores the physical quantity observed by the state observation unit [A] as data;
a reward condition setting unit [C] that sets reward conditions in machine learning;
a reward calculation unit [D] that calculates a reward based on the data of the physical quantity observed by the state observation unit [A] and the reward condition set by the reward condition setting unit [C];
a production condition learning unit [E] that performs machine learning for adjusting production conditions based on the remuneration calculated by the remuneration calculation unit [D], the physical quantity data, and the production conditions set in the metal material production unit;
A learned condition storage unit [F] for storing learning results obtained by machine learning in the production condition learning unit [E];
a production condition output unit [G] that determines and outputs adjustment amounts of production conditions for each facility that constitutes the metal material production unit based on the learning result of the production condition learning unit [E]; ,
At least one of the physical quantity data storage unit [B], the remuneration condition setting unit [C], the remuneration calculation unit [D], the production condition learning unit [E], and the learned condition storage unit [F] , input as external data consisting of the surface condition, shape, material strength and bending workability of the metal material measured using the metal material manufactured in the metal material production department, and the learning of the production condition learning department [E] A metal material production system characterized by being used for 前記学習済み記憶部［Ｆ］に記憶された学習結果を前記生産条件学習部［Ｅ］の学習に使用することを特徴とする、請求項１に記載の金属材料生産システム。 2. The metal material production system according to claim 1 , wherein the learning result stored in said learned storage unit [F] is used for learning of said production condition learning unit [E]. 前記生産条件学習部［Ｅ］で学習した結果を、前記生産条件出力部［Ｇ］に反映させて、前記金属材料生産部の制御ユニットに指示を出すことを特徴とする、請求項１または２に記載の金属材料生産システム。 3. A control unit of said metal material production section is instructed by reflecting the result learned by said production condition learning section [E] in said production condition output section [G]. The metal material production system according to . 前記報酬計算部［Ｄ］は、前記物理量のデータにあらかじめ許容範囲が設定され、前記物理量のデータの数値が、前記許容範囲内に収まると、プラスの報酬を与えるように算出することを特徴とする、請求項１～３のいずれか１つに記載の金属材料生産システム。 The remuneration calculation unit [D] is characterized in that a permissible range is set in advance for the data of the physical quantity, and when the numerical value of the data of the physical quantity falls within the permissible range, calculation is performed so as to give a positive remuneration. The metal material production system according to any one of claims 1 to 3 . 前記報酬計算部［Ｄ］は、前記物理量のデータにあらかじめ許容範囲値が設定され、前記物理量のデータの数値が、前記許容範囲外になると、前記許容範囲の限界値からの前記物理量のデータの数値のずれ幅に応じてマイナスの報酬を与えるように算出することを特徴とする、請求項１～４のいずれか１つに記載の金属材料生産システム。 In the remuneration calculation unit [D], an allowable range value is set in advance for the data of the physical quantity. 5. The metal material production system according to any one of claims 1 to 4 , wherein calculation is performed so as to give a negative reward according to the range of deviation of the numerical values. 金属材料を生産するに際し、
状態観測部［Ａ］により、実行中の金属材料生産に関する物理量を観測する工程と、
物理量データ記憶部［Ｂ］に前記物理量をデータとして記憶する工程と、
報酬条件設定部［Ｃ］により、機械学習における報酬条件を設定する工程と、
報酬計算部［Ｄ］により、前記状態観測部［Ａ］で観測した前記物理量のデータ、および前記報酬条件設定部［Ｃ］で設定された前記報酬条件に基づいて報酬を算出する工程と、
生産条件学習部［Ｅ］により、前記報酬計算部［Ｄ］が算出した前記報酬、前記物理量データ、および前記金属材料生産部で設定されている生産条件に基づいて生産条件調整の機械学習を行う工程と、
学習済み条件記憶部［Ｆ］に前記生産条件学習部［Ｅ］で機械学習した学習結果を記憶する工程と、
生産条件出力部［Ｇ］により、前記生産条件学習部［Ｅ］での前記学習結果に基づいて、前記金属材料生産部を構成する各設備の生産条件の調整量を決定して出力する工程と、を備え、
前記物理量データ記憶部［Ｂ］、前記報酬条件設定部［Ｃ］、前記報酬計算部［Ｄ］、前記生産条件学習部［Ｅ］および前記学習済み条件記憶部［Ｆ］の少なくともいずれか１つに、前記金属材料生産部で製造された金属材料を用いて測定した金属材料の表面状態、形状、材料強度および曲げ加工性からなる外部データとして入力し、前記生産条件学習部［Ｅ］の学習に使用することを特徴とする、金属材料生産方法。
When producing metal materials,
A step of observing physical quantities related to the metal material production being executed by the state observation unit [A];
a step of storing the physical quantity as data in the physical quantity data storage unit [B];
A step of setting a reward condition in machine learning by a reward condition setting unit [C];
a step of calculating a reward by a reward calculation unit [D] based on the data of the physical quantity observed by the state observation unit [A] and the reward condition set by the reward condition setting unit [C];
The production condition learning unit [E] performs machine learning for production condition adjustment based on the remuneration calculated by the remuneration calculation unit [D], the physical quantity data, and the production conditions set in the metal material production unit. process and
a step of storing the result of machine learning performed by the production condition learning unit [E] in the learned condition storage unit [F];
a step of determining and outputting the adjustment amount of the production conditions of each facility constituting the metal material production unit by the production condition output unit [G] based on the learning result of the production condition learning unit [E]; , and
At least one of the physical quantity data storage unit [B], the remuneration condition setting unit [C], the remuneration calculation unit [D], the production condition learning unit [E], and the learned condition storage unit [F] , input as external data consisting of the surface condition, shape, material strength and bending workability of the metal material measured using the metal material manufactured in the metal material production department, and the learning of the production condition learning department [E] A metal material production method characterized by being used for
JP2019001383A 2019-01-08 2019-01-08 Metal material production system and metal material production method Active JP7189777B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2019001383A JP7189777B2 (en) 2019-01-08 2019-01-08 Metal material production system and metal material production method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2019001383A JP7189777B2 (en) 2019-01-08 2019-01-08 Metal material production system and metal material production method

Publications (2)

Publication Number Publication Date
JP2020112882A JP2020112882A (en) 2020-07-27
JP7189777B2 true JP7189777B2 (en) 2022-12-14

Family

ID=71667977

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2019001383A Active JP7189777B2 (en) 2019-01-08 2019-01-08 Metal material production system and metal material production method

Country Status (1)

Country Link
JP (1) JP7189777B2 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011258113A (en) 2010-06-11 2011-12-22 Kobe Steel Ltd Manufacturing condition determination system for metal band material
US20130178952A1 (en) 2010-06-28 2013-07-11 Precitec Itm Gmbh Method for closed-loop controlling a laser processing operation and laser material processing head using the same
JP2017030152A (en) 2015-07-28 2017-02-09 ファナック株式会社 Injection molding system capable of calculating optimum operating conditions, and machine learning apparatus
JP2017042882A (en) 2015-08-27 2017-03-02 ファナック株式会社 Wire electric discharge machine for working while adjusting the working condition

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06145930A (en) * 1992-11-02 1994-05-27 Furukawa Electric Co Ltd:The Production of precipitation type copper alloy
JPH09258804A (en) * 1996-03-21 1997-10-03 Kobe Steel Ltd Operating condition guidance device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011258113A (en) 2010-06-11 2011-12-22 Kobe Steel Ltd Manufacturing condition determination system for metal band material
US20130178952A1 (en) 2010-06-28 2013-07-11 Precitec Itm Gmbh Method for closed-loop controlling a laser processing operation and laser material processing head using the same
JP2017030152A (en) 2015-07-28 2017-02-09 ファナック株式会社 Injection molding system capable of calculating optimum operating conditions, and machine learning apparatus
JP2017042882A (en) 2015-08-27 2017-03-02 ファナック株式会社 Wire electric discharge machine for working while adjusting the working condition

Also Published As

Publication number Publication date
JP2020112882A (en) 2020-07-27

Similar Documents

Publication Publication Date Title
JP6292309B2 (en) Rolling simulation device
JP6297159B2 (en) Method for producing metal steel alloys and / or iron alloys in hot and thick plate mills using optimized structure simulators, structure monitors and / or structure models
CN101765467B (en) Method for adjusting a state of a rolling stock, particularly a near-net strip
CN109308048B (en) Machining machine system and manufacturing system
CN103212585B (en) A control device of a hot mill used for thin plates and a control method of the hot mill used for thin plates
KR102545735B1 (en) How to operate a continuous processing line
JPWO2006040823A1 (en) Material control method and apparatus for rolling, forging or straightening line
EP3926425A1 (en) Method for determining setting conditions of manufacturing facility, method for determining mill setup setting value of rolling mill, device for determining mill setup setting value of rolling mill, method for manufacturing manufactured object, and method for manufacturing rolled stock
US5778151A (en) Method and control device for controlling a material-processing process
JP7189777B2 (en) Metal material production system and metal material production method
JP2021536063A (en) Methods and electronic devices for monitoring the manufacture of metal products, related computer programs and equipment
JP7233224B2 (en) Metal material production system and metal material production method
JP2020157327A (en) Control method for outlet side temperature of finished steel sheet, control device therefor and production method for steel sheet
WO2022054500A1 (en) System for predicting material characteristic value, and method for producing metal sheet
JP4627371B2 (en) Al alloy plate material prediction method
JP2009142879A (en) Temper rolling method
JP7287416B2 (en) Thick steel plate manufacturing specification determination support device, manufacturing specification search method, computer program, computer-readable recording medium, and thick steel plate manufacturing method
JPH042957A (en) Apparatus for steel plate component prediction
JPH044911A (en) Method for predicting the quality of steel material
KR20220134042A (en) Method, control system and production line for controlling the flatness of a strip of rolled material
Agarwal et al. Process-structure-microstructure relationship in hot strip rolling of steels using statistical data mining
JP5007630B2 (en) Product quality control method and control device
KR20080022074A (en) Rolling line material quality prediction and control apparatus
JP2015199123A (en) Control method and control method of temper rolling equipment
Reeve et al. Control, automation and the hot rolling of steel

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20210922

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20220729

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20220809

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20220929

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20221108

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20221202

R151 Written notification of patent or utility model registration

Ref document number: 7189777

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151