WO2020130513A1 - System and method for predicting material properties using metal microstructure images based on deep learning - Google Patents

Abstract

The present invention provides a system for predicting material properties and a method according thereto, the system for predicting material properties comprising: a deep learning device for applying deep learning to microstructure images of metals and steel grade and property information of the metals, and building a prediction model for predicting the steel grade and property information of a metal according to a microstructure image; and an interface device for receiving a microstructure image of a metal of which the steel grade and property information is unknown, providing the received microstructure image to the prediction model, and acquiring, from the prediction model, steel grade and property information corresponding to the microstructure image.

Description

System and method for predicting material properties using metal microstructure images based on deep learning

The present invention relates to a system and method for predicting material properties using metal microstructure images based on deep learning.

In the past, material properties, that is, mechanical properties (hardness, tensile strength, yield strength, elongation, etc.) are predicted for the composition of materials, manufacturing/processing methods (rolling temperature, cooling rate, rolling reduction, heat treatment conditions, etc.), microstructure ( Prediction is carried out by mathematical modeling methods such as transformation amount, crystal grain size, phase fraction, dendritic interlayer spacing measurement, and so on. However, due to the inaccuracy and difficulty of factor measurement, it is difficult to generalize because it can be used only under certain conditions of some materials, not mathematical modeling that can be applied universally to all materials.

Here, the image pattern of the microstructure that determines the mechanical properties of the material is determined according to the material composition and manufacturing/processing method. The types of microstructure phases academically separated include pearlite, austenite, ferrite, martensite, bainite, and the like.

The image of the microstructure is called by the same name even if the type of material is different. For example, even if it is called the same pearlite, if the material is different, its image pattern will be different. That is, this is the reason why the mathematical prediction model used in the past cannot be generalized.

In this regard, Patent Document 1 (Registration Patent No. 10-0032944 (2014.03.20): name of the invention-a method for predicting metal properties) discloses a method for predicting metal properties, and the disclosed method, in particular, a low pressure casting method Metal property prediction method for easily predicting the mechanical properties of an aluminum alloy used in an aluminum wheel manufactured by the method, generating a property conversion equation according to the aluminum alloy material, and setting the importance for the generated property conversion equation It consists of inputting one of the mechanical property values of the mechanical property value, and calculating the remaining mechanical property value using the input mechanical property value and the generated physical property conversion equation. On the other hand, the step of setting the importance is set in the order of the smallest error range, and the mechanical property values are secondary resin assumed interval (SDAS), yield strength, tensile strength, Vickers hardness, Brinell hardness, fatigue strength, impact energy, and elongation. .

In addition, Patent Document 2 (Registration Patent No. 10-0920626 (2009.09.29): the name of the invention-a method for predicting the mechanical properties of a steel material cooled after rolling) discloses a method for predicting the mechanical properties of a steel material cooled after rolling, According to the disclosed invention, the step of obtaining the transformation amount (X1) during air cooling of the steel material cooled after rolling, the step of obtaining the transformation amount (X2) during water cooling of the steel material cooled after rolling, the particle diameter of ferrite transformed during air cooling and water cooling A method for predicting the mechanical properties of a steel material that is cooled after rolling is provided, each comprising obtaining an average particle diameter from the ferrite particle diameter, and obtaining the tensile strength and yield strength of the steel from the average particle diameter and hardness obtained from the average transformation temperature. . According to this, even if the cooling rate changes during cooling after rolling the steel, it provides an effect of accurately measuring the transformation behavior and predicting the mechanical properties of the steel over time, thereby controlling the properties of the final microstructure. .

It is an object of the present invention to provide a system and method for predicting physical properties from a microscopic image using a feature extraction method of a microscopic image by deep learning without relying on a conventional academic microstructure classification method. .

In order to solve the above-described problems, an aspect of the present invention provides a material property prediction system. The material property prediction system includes: a deep learning device that deeply learns a microstructure image of a metal, a steel type and physical property information of a metal, and builds a prediction model for predicting a steel type and physical property information of a metal according to the microstructure image; And an interface device that receives a microstructure image of a metal whose steel type and physical property information is unknown, provides an input microstructure image to a prediction model, and obtains steel type and physical property information corresponding to the microstructure image from the prediction model. .

In one embodiment, the material property prediction system may further include an image pre-processing device that pre-processes the micro-tissue image of the metal learned in the deep learning device, and the deep learning device predicts the micro-structure image of the pre-treated metal. Build a model.

In one embodiment, the interface device may pre-process the input micro-tissue image and obtain steel grade and physical property information from the prediction model based on the pre-processed micro-tissue image.

In one embodiment, the property information may include at least one of hardness, tensile strength, yield strength, and elongation.

In one embodiment, the deep learning apparatus includes an effective region deep learning module that deep-learns an effective region to be analyzed in a micro-structured image of a metal, and builds a predictive model according to the effective region of the micro-structured image.

In one embodiment, the deep learning device includes an image type deep learning module that deep-learns the image type of a microstructure image of metal, and can build a predictive model according to the image type.

Another aspect of the present invention provides a method for predicting material properties. The material property prediction method includes: deep learning a microstructure image of a metal, a steel type of a metal, and property information; Constructing a prediction model for predicting metal grade and physical property information according to the microstructure image; And obtaining steel type and physical property information corresponding to the microstructure image of the metal whose steel type and physical property information is unknown from a prediction model.

In one embodiment, the property information may include at least one of hardness, tensile strength, yield strength, and elongation.

In one embodiment, the method for predicting material properties may further include preprocessing a microstructure image of a learned metal, and the deep learning step may construct a prediction model using a microstructure image of the pretreated metal. Can.

In one embodiment, the step of obtaining from the predictive model includes preprocessing the microstructure image of the metal whose steel type and physical property information is unknown, and obtaining steel type and physical property information corresponding to the pretreated microstructure image from the predictive model. can do.

In one embodiment, the step of deep learning may include the step of deep learning the effective region to be analyzed in the microstructure image of the metal, and constructing a predictive model according to the effective region of the microstructure image.

In one embodiment, the step of deep learning may include the step of deep learning the image type of the microstructure image of the metal and constructing a predictive model according to the image type.

In addition, the summary of the said invention does not list all the characteristics of this invention. In addition, subcombinations of this feature group can also be an invention.

According to the present invention, it is possible to automatically and easily predict the physical properties of a metal from the microstructure image of the metal.

1 is a schematic block diagram of a material property prediction system according to an embodiment of the present invention.

2 is a schematic flowchart of a method for predicting material properties according to an embodiment of the present invention.

Figure 3 shows a microstructure image for each heat treatment condition of the sample used in an embodiment of the present invention.

Figure 4 schematically shows a method for producing a patch image for learning used in an embodiment of the present invention.

5 is a graph showing predicted physical property results according to an embodiment of the present invention.

6 is a diagram illustrating a computer environment corresponding to a material property prediction system according to an embodiment of the present invention.

Hereinafter, a material property prediction system and a material property prediction method according to an embodiment of the present invention will be described with reference to the drawings.

1 is a schematic block diagram of a material property prediction system 100 according to an embodiment of the present invention. As shown in FIG. 1, the material property prediction system 100 includes a deep learning device 110 and an interface device 120.

The deep learning device 110 deep-learns the microstructure image of metal, the steel type and physical property information of the metal, and constructs a prediction model for predicting the steel type and physical property information of the metal according to the microstructure image. In addition, the interface device 120 receives a microstructure image of a metal whose steel type and physical property information is unknown, provides an input microstructure image to a prediction model, and predicts steel type and physical property information corresponding to the microstructure image. Obtained from the model.

In the material property prediction system 100 according to an embodiment of the present invention, in order to build a predictive model, first, a metal material is prepared under various manufacturing conditions and heat treatment conditions to obtain a sample, and then a microstructure image of the metal material Prepare. For example, the microstructure image may be obtained according to various magnification and etching conditions using an optical microscope or an electron microscope. In addition, it is prepared by measuring the property information for the sample, for example, hardness, tensile strength, yield strength and/or elongation of the sample. One micro-tissue image may be obtained from one sample, or a plurality of micro-tissue images may be obtained from one sample.

On the other hand, in one embodiment, the material property prediction system 100 according to an embodiment of the present invention further includes an image pre-processing device 130 for pre-processing the microstructure image of the metal learned in the deep learning device 110 Can. At this time, the deep learning device 110 builds a predictive model using the microstructure image of the pre-treated metal.

For example, the image pre-processing device 130 may resize the micro-tissue image to have the same resolution or normalize the image before inputting the micro-tissue image to the deep learning device 110. Accordingly, the deep learning device 110 may improve the accuracy of the prediction model by constructing the prediction model using the same or similar resolution or normalized microscopic tissue images.

In this case, in one embodiment, the interface device 120 may pre-process the input micro-tissue image and obtain steel type and physical property information from the predictive model based on the pre-processed micro-tissue image. The pre-processing performed by the interface device 120 may correspond to the pre-processing of the image pre-processing device 130 described above. Therefore, the prediction accuracy can be further improved by predicting the steel type and physical property information through the predictive model after pre-processing the input micro-tissue image corresponding to the pre-processing method used to construct the predictive model.

Meanwhile, the deep learning device 110 includes an effective area deep learning module 112 that deep-learns an effective area to be analyzed in a microstructure image of a metal, and builds a predictive model according to the effective area of the microstructure image. have. For example, the input micro-tissue image may include scale information or other text information added when the image is generated, and since such information is not related to the physical properties of the corresponding sample, deep learning of the micro-tissue image is performed. This can affect accuracy. Accordingly, the effective area deep learning module 112 learns an effective area effective for deep learning from a previously input image, provides only the learned effective area, and the deep learning device 110 predicts only the learned effective area. Used to build. Therefore, the accuracy of the prediction model can be further improved.

In addition, the deep learning device 110 includes an image type deep learning module 114 that deep-learns the image type of the microstructure image of metal, and can build a predictive model according to the image type. For example, since microscopic images obtained from an optical microscope and microscopic images acquired from an electron microscope have different properties, it is preferable to predict physical properties by classifying image types. To this end, the image type deep learning module 114 receives the input micro-tissue image and the image type of the corresponding image, learns the image type of the micro-tissue image, and the deep learning device 110 performs physical properties according to the learned image type. Predictive models can be built to predict information. Therefore, the accuracy of the prediction model can be further improved.

When the predictive model constructed as described above is constructed, the interface device 120 receives a microstructure image of a metal whose steel type and physical property information is unknown as described above, and the steel type and physical properties corresponding to the microstructure image input from the predictive model. Since information can be acquired, steel type and physical property information of a corresponding metal material can be obtained only by inputting the microstructure image.

Next, a method for predicting material properties according to an embodiment of the present invention will be described with reference to FIG. 2. 2 is a flowchart of a method 200 for predicting material properties according to an embodiment of the present invention.

First, the method 200 for predicting material properties according to an embodiment of the present invention starts from deep learning (S210) of a microstructure image of a metal, a steel type of a metal, and property information. For example, a metal material is prepared under various manufacturing conditions and heat treatment conditions to obtain a sample, and then a microstructure image of the metal material is prepared. For example, the microstructure image may be obtained according to various magnification and etching conditions using an optical microscope or an electron microscope. In addition, it is prepared by measuring the property information for the sample, for example, hardness, tensile strength, yield strength and/or elongation of the sample. One micro-tissue image may be obtained from one sample, or a plurality of micro-tissue images may be obtained from one sample.

In one embodiment, in step S210, the microstructure image of the metal to be learned is pre-processed, the pre-processed microstructure image, and the steel type and physical property information of the metal may be deep-learned. For example, before deep-learning a microstructure image of a metal, a steel grade, and property information of a metal, the microstructure image may be preprocessed by resizing the microstructure image to have the same resolution or normalizing the image. By constructing a prediction model as described below using the same or similar resolution or normalized micro-tissue images, the accuracy of the prediction model can be improved.

On the other hand, in step S210, property information about the sample, for example, hardness, tensile strength, yield strength and/or elongation of the sample is measured and prepared. In addition, one micro-tissue image may be obtained from one sample, or a plurality of micro-tissue images may be obtained from one sample.

Next, a predictive model for predicting metal steel type and physical property information according to the microstructure image is constructed (S220 ). For example, a predictive model can be built by deep learning microstructure images and steel grade and property information by regression analysis.

Next, steel type and physical property information corresponding to the microstructure image of the metal in which the steel type and physical property information are unknown are obtained from the prediction model (S230). In one embodiment, in step S230, the input micro-tissue image is pre-processed, and steel grade and physical property information may be obtained from the predictive model based on the pre-processed micro-tissue image. The pre-processing performed in step S230 may correspond to the pre-processing performed in step S210 described above. Therefore, the prediction accuracy can be further improved by predicting the steel type and physical property information through the predictive model after pre-processing the input micro-tissue image corresponding to the pre-processing method used to construct the predictive model.

On the other hand, in step S210, the effective region to be analyzed in the microstructure image of the metal may be deep-learned, and in step S220, a predictive model may be constructed according to the effective region of the microstructure image. For example, the input micro-tissue image may include scale information or other text information added when the image is generated, and since such information is not related to the physical properties of the corresponding sample, deep learning of the micro-tissue image is performed. This can affect accuracy. Therefore, an effective region effective for deep learning is learned from the image previously input in step S210, only the learned effective region can be deep-learned, and in step S220, the learned effective region can be used to build a prediction model. have. Therefore, the accuracy of the prediction model can be further improved.

In addition, in step S210, the image type of the microstructure image of the metal may be deep-learned, and in step S220, a predictive model may be constructed according to the image type. For example, since microscopic images obtained from an optical microscope and microscopic images acquired from an electron microscope have different properties, it is preferable to predict physical properties by classifying image types. To this end, in step S210, the image type of the micro-tissue image is deep-learned, and in step S220, a prediction model can be constructed so that physical property information can be predicted for each type of the learned image. Therefore, the accuracy of the prediction model can be further improved.

When the predictive model constructed as described above is constructed, in step S230, steel type and physical property information corresponding to the input microscopic structure image may be obtained from the microstructure image of the metal in which the steel type and physical property information are unknown, as described above. With only the microstructure image, it is possible to obtain steel type and physical property information of a corresponding metal material.

Hereinafter, the results of predicting the material properties using the material property prediction system 100 of the present invention will be described. The inventor performed heat treatment under three conditions using trip steel (TRIP), a type of ultra high tensile steel for automobiles, and measured tensile strength, yield strength, and elongation of the heat treated trip steel. The mechanical properties data according to the heat treatment conditions of the trip steel are as follows.

Heat treatment type		Mechanical properties
		Hardness (HV)	Yield strength (YS, MPa)	Tensile strength (TS, MPa)	Elongation (EL, %)
One	TRIP-250-450	431	1187	1274	14.4
2	TRIP-310-450	417	1111	1247	17.0
3	TRIP-450-450	466	942	1476	9.9

Next, an image of the microstructure of the trip steel thus heat treated was obtained using an electron microscope. Figure 3 shows the microstructure image for each heat treatment condition. At this time, as for the microstructure image, 90 images having a resolution of 2560x1920 were prepared for each heat treatment condition. Next, in order to secure an image to be used for constructing a predictive model, data augmentation is performed by creating a patch image having a resolution of 800x800 by applying a window of 800x800 to a microstructure image having a resolution of 2560x1920. Thus, 900 patch images were obtained for each heat treatment condition. Accordingly, 2,700 learning images are used in the deep learning device 110. 4 schematically shows a method of producing a patch image for learning. Then, for deep learning of regression analysis, a patch image of 800x800 resolution was converted back to a resolution of 227x227.

Next, the patch image for each heat treatment condition was converted into a matrix of 4D arrays, and a predictive model was constructed by performing deep learning on the regression analysis by matching the material properties of Table 1 to the converted matrix.

In the predictive model constructed as described above, input of the microstructure of the original was input, and a resized and normalized image was generated at a resolution of 227x227 through a pre-processing process to predict physical properties. 5 is a graph showing the results of predicting physical properties. As shown in FIG. 5, on the basis of the non-learned microstructure image, hardness, yield strength, and tensile strength showed an accuracy of approximately 90%, and elongation showed an accuracy of approximately 79%.

6 is a diagram illustrating a computer environment corresponding to a material property prediction system according to an embodiment of the present invention. Referring to FIG. 3, an illustration of a system 1000 including a computing device 1100 configured to implement one or more of the embodiments described above is shown. For example, computing device 1100 may be a personal computer, server computer, handheld or laptop device, mobile device (mobile phone, PDA, media player, etc.), multiprocessor system, consumer electronics, mini computer, mainframe computer, Distributed computing environments including any of the aforementioned systems or devices, and the like, but is not limited thereto.

The system 1000 may be an example of a material property prediction system according to an embodiment of the present invention described above. In addition, the method for predicting material properties according to an embodiment of the present invention described above may be implemented using the system 1000.

Computing device 1100 may include at least one processing unit 1110 and memory 1120. Here, the processing unit 1110 may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application specific integrated circuit (ASIC), or field programmable gate arrays (FPGA). And may have multiple cores. The memory 1120 may be volatile memory (eg, RAM, etc.), non-volatile memory (eg, ROM, flash memory, etc.) or a combination thereof.

Further, the computing device 1100 may include additional storage 1130. The storage 1130 includes, but is not limited to, magnetic storage, optical storage, and the like. Computer-readable instructions for implementing one or more embodiments disclosed herein may be stored in the storage 1130, and other computer-readable instructions for implementing an operating system, application programs, and the like may also be stored. Computer readable instructions stored in storage 1130 may be loaded into memory 1120 for execution by processing unit 1110. The processing unit 1110 and the storage 1130 may respectively correspond to a calculation module and a storage module according to an embodiment of the present invention.

Further, the computing device 1100 may include an input device(s) 1140 and an output device(s) 1150. Here, the input device(s) 1140 may include, for example, a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device, or any other input device. Also, the output device(s) 1150 may include, for example, one or more displays, speakers, printers, or any other output device. Also, the computing device 1100 may use an input device or output device provided in another computing device as the input device(s) 1140 or the output device(s) 1150.

In addition, computing device 1100 may include communication connection(s) 1160 that enable computing device 1100 to communicate with other devices (eg, computing device 1300 ). Here, the communication connection(s) 1160 may be a modem, network interface card (NIC), integrated network interface, radio frequency transmitter/receiver, infrared port, USB connection, or other for connecting the computing device 1100 to another computing device. Interface. Further, the communication connection(s) 1160 may include a wired connection or a wireless connection.

Each component of the above-described computing device 1100 may be connected by various interconnections such as a bus (eg, peripheral component interconnection (PCI), USB, firmware (IEEE 1394), optical bus structure, etc.) Or may be interconnected by the network 1200.

As used herein, terms such as "component", "module", "system", "interface" and the like generally refer to a computer-related entity that is hardware, a combination of hardware and software, software, or running software. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. For example, both the application running on the controller and the controller may be components. One or more components can reside within a thread of process and/or execution, and components can be localized on one computer or distributed between two or more computers.

In the above, the present invention has been described using examples, but the technical scope of the present invention is not limited to the ranges described in the above examples. It is apparent to those skilled in the art to which the present invention pertains that various modifications or improvements can be added to the above embodiments. It is clear from the description of the claims that the form in which such a change or improvement is added may be included in the technical scope of the present invention.

[Description of codes]

100 material properties prediction system

110 deep learning device

112 Effective Area Deep Learning Module

114 Image Type Deep Learning Module

120 interface device

130 image pre-processing unit

1100 computing device

1110 processing unit

1120 memory

1130 storage

1140 input device

1150 output device

1160 communication connection

Claims (12)

1. A deep learning device that deeply learns the microstructure image of metal, the steel type and physical property information of the metal, and builds a predictive model predicting the steel type and physical properties of the metal according to the microstructure image; And

   An interface device that receives a microstructure image of a metal whose steel type and physical property information is unknown, provides the input microstructure image to the prediction model, and obtains steel type and physical property information corresponding to the microstructure image from the prediction model

   Containing,

   Material property prediction system.
2. According to claim 1,

   The property information includes at least one of hardness, tensile strength, yield strength and elongation,

   Material property prediction system.
3. According to claim 1,

   Further comprising an image pre-processing device for pre-processing the micro-tissue image of the metal learned in the deep learning device, the deep learning device to build the predictive model using the micro-structured image of the pre-treated metal,

   Material property prediction system.
4. According to claim 3,

   The interface device pre-processes the inputted micro-tissue image, and acquires steel type and physical property information from the prediction model based on the pre-processed micro-tissue image,

   Material property prediction system.
5. According to claim 1,

   The deep learning device includes an effective area deep learning module that deep-learns an effective area to be analyzed in a microstructure image of metal, and constructs the prediction model according to the effective area of the microstructure image,

   Material property prediction system.
6. According to claim 1,

   The deep learning device includes an image type deep learning module for deep learning an image type of a microstructure image of metal, and constructing the prediction model according to the image type,

   Material property prediction system.
7. Deep-learning a microstructure image of a metal, steel type and physical property information of the metal;

   Constructing a prediction model predicting steel type and physical property information of the metal according to the microstructure image; And

   Obtaining steel grade and physical property information from the prediction model corresponding to a microstructure image of a metal whose steel type and physical property information is unknown.

   Containing,

   How to predict material properties.
8. The method of claim 7,

   The property information includes at least one of hardness, tensile strength, yield strength and elongation,

   How to predict material properties.
9. The method of claim 7,

   Further comprising the step of pre-processing the microstructure image of the metal to be learned, the deep learning step, using the pre-structured microstructure image of the metal to build the predictive model,

   How to predict material properties.
10. The method of claim 9,

    The step of obtaining from the predictive model includes pre-processing a microstructure image of a metal whose steel type and physical property information is unknown, and obtaining steel type and physical property information corresponding to the pre-processed microstructure image from the predictive model,

    How to predict material properties.
11. The method of claim 7,

    The deep learning includes deep learning an effective region to be analyzed in a microstructure image of a metal, and constructing the prediction model according to the effective region of the microstructure image,

    How to predict material properties.
12. The method of claim 7,

    The deep learning step includes deep learning an image type of a microstructure image of a metal and constructing the prediction model according to the image type.

    How to predict material properties.

Images