JP7468481B2 - Method for generating model for estimating meandering amount of steel plate, method for estimating meandering amount, and manufacturing method - Google Patents

Description

The present invention relates to a method for generating a model for estimating the amount of meandering of a steel plate, a method for estimating the amount of meandering, and a manufacturing method.

In the steel plate manufacturing process, the steel plate is often transported to the next process as a steel strip coil, which is a coiled steel plate wound in an intermediate process. In this case, the transported steel strip coil is unloaded from the pay-off reel of the steel plate manufacturing line, and is subjected to various processing processes such as processing, heat treatment, coating, and inspection, and then re-formed into a steel strip coil by a winder. Between the pay-off reel and the winder, the steel plate is continuously transported while being in contact with various rolls such as a pinch roll that holds the steel plate with a pair of rolls, a bridle roll that applies tension to the steel plate in a specified section, a deflector roll that adjusts the direction of travel of the steel plate, and a looper roll that adjusts the processing timing of the steel plate. Such equipment is called a continuous transport equipment. At this time, the center position of the steel plate in the width direction may shift from the center position of the width direction of the manufacturing line (line center), and a meandering phenomenon may occur in which the steel plate moves to the width direction end side of the roll. When the meandering phenomenon occurs, the passing property of the steel plate becomes unstable, so the processing speed of the steel plate decreases, and the production efficiency of the manufacturing line may decrease.

In particular, in continuous steel sheet conveying equipment, if the interval between when the steel sheet comes into contact with a roll and when it comes into contact with the next roll is long, the effect of the rolls in restraining the movement of the steel sheet toward the widthwise end of the roll is reduced, and the meandering phenomenon tends to occur easily. Examples of production lines in which the meandering phenomenon is a problem include continuous annealing lines and continuous hot-dip galvanizing lines. In continuous annealing lines and continuous hot-dip galvanizing lines, heat treatment is performed to adjust the material properties of the steel sheet that has been work-hardened in the cold rolling process. The heat treatment in the continuous conveying equipment involves temperature changes of the steel sheet, such as heating the steel sheet, holding it at a high temperature, and cooling it, and the tension applied to the steel sheet at this time is relatively small. Furthermore, since the distance between the conveying rolls is long, the conditions are such that the meandering phenomenon is more likely to occur.

In order to suppress the occurrence of the meandering phenomenon and stably produce steel sheets, it is necessary to accurately measure the meandering amount of the steel sheet in the continuous conveying equipment and appropriately control the meandering amount based on the measured meandering amount. Here, an example of a method for controlling the meandering amount is a method of changing the inclination angle of the conveying roll (called steering). In addition, as a method for measuring the meandering amount, there is a method of detecting the width direction end position (hereinafter also referred to as the edge position) of the steel sheet by a backlight method in which a scanning type detector is installed above the steel sheet and a light source is provided on the lower side of the steel sheet. On the other hand, a method for measuring the meandering amount using a two-dimensional CCD camera has also been proposed. For example, Patent Document 1 relates to a method for measuring the camber (plate bending) of a steel sheet in the rough rolling process of a hot rolling line, and describes a method in which a two-dimensional image sensor such as a CCD camera is installed above the exit side of the rough rolling mill and line scanning is performed on the acquired image to calculate the meandering amount of the steel sheet.

Patent Document 2 also describes a means for measuring the amount of meandering of a steel sheet in a continuous annealing furnace. Specifically, Patent Document 2 describes taking an image including the rolls and steel sheet in the continuous annealing furnace with a camera, and sending the captured image data to an image processing device. As an image processing method for this, for example, a method is described in which monochrome image data is converted into digital data of about 8 bits, noise is removed, and then binarization is performed to extract feature quantities such as edge positions. It also describes measuring the distance between the line center and the edge position based on the positional relationship between the steel sheet image and the in-furnace roll image after binarization, and calculating the amount of meandering taking into account the sheet width of the steel sheet.

Japanese Patent Application Laid-Open No. 4-167911 Japanese Patent Application Laid-Open No. 5-331555

However, the method for measuring the amount of meandering described in Patent Document 1 performs line scanning of one scanning line in the width direction of the steel plate at one point in the conveying direction of the steel plate to identify the edge position, and calculates the amount of meandering based on the result. Therefore, if there are mist-like water droplets or fumes around the steel plate, the edge position may not be identified, and an abnormal value may be output as the measurement result of the amount of meandering. On the other hand, the method for measuring the amount of meandering described in Patent Document 2 requires that a threshold value for performing the binarization process (a boundary value for binarizing 8-bit digital data in the above example) be appropriately set in advance in order to perform binarization processing of the image data. However, an appropriate threshold value changes depending on the lighting conditions when the image data is acquired by the camera, the brightness and color tone of the steel plate to be imaged, the presence or absence of steam or disturbances in the vicinity, and the environment in which the image data is acquired. For this reason, it is necessary for a person to finely adjust the threshold value according to changes in the shooting environment. Furthermore, due to dirt on the camera lens, changes in the color tone of the equipment, etc. that are imaged as the background of the steel plate, and changes in the color tone of the steel plate during the heat treatment process, the captured image may vary, and the edge position may not be clearly extracted. For these reasons, there was a desire to provide technology that could accurately estimate the amount of steel plate meandering.

The present invention has been made to solve the above problems, and aims to provide a method for generating a meandering amount estimation model for a steel plate, which is capable of generating a meandering amount estimation model that accurately estimates the meandering amount of a steel plate. Another object of the present invention is to provide a method for estimating the meandering amount of a steel plate, which is capable of accurately estimating the meandering amount of a steel plate. Another object of the present invention is to provide a manufacturing method for a steel plate, which is capable of suppressing the occurrence of manufacturing problems due to meandering of the steel plate and manufacturing steel plate with a good yield.

The method for generating a meandering amount estimation model for a steel plate according to the present invention includes a data acquisition step of acquiring image data of the steel plate by photographing the moving steel plate and measurement data of the meandering amount of the steel plate corresponding to the image data, and a model generation step of generating a meandering amount estimation model for estimating the meandering amount of the steel plate by machine learning using a convolutional neural network technique using multiple learning data, with the image data acquired in the data acquisition step being input performance data and the measurement data of the meandering amount of the steel plate corresponding to the image data being output performance data.

The convolutional neural network may include one or more average pooling layers upstream of the convolutional layers.

The number of channels of the average pooling layer is preferably the same as the number of channels of the image data that is the input data of the meandering amount estimation model.

The method for estimating the meandering amount of a steel plate according to the present invention includes a step of estimating the meandering amount of the steel plate by inputting image data obtained by photographing a moving steel plate to a meandering amount estimation model generated by the method for generating a meandering amount estimation model of a steel plate according to the present invention.

The method for manufacturing a steel sheet according to the present invention includes a step of controlling the amount of meandering of a steel sheet in a continuous annealing facility using the method for estimating the amount of meandering of a steel sheet according to the present invention.

According to the method for generating a model for estimating the meandering amount of a steel plate according to the present invention, it is possible to generate a model for estimating the meandering amount of a steel plate that accurately estimates the meandering amount of the steel plate. Furthermore, according to the method for estimating the meandering amount of a steel plate according to the present invention, it is possible to accurately estimate the meandering amount of the steel plate. Furthermore, according to the method for manufacturing a steel plate according to the present invention, it is possible to suppress the occurrence of manufacturing problems due to meandering of the steel plate and manufacture the steel plate with a good yield.

FIG. 1 is a schematic diagram showing the configuration of a continuous steel sheet conveying facility according to one embodiment of the present invention. FIG. 2 is a diagram showing an example of image data of a steel plate and installation positions of cameras. FIG. 3 is a block diagram showing the configuration of the data acquisition unit and the meandering amount estimation model generation unit. FIG. 4 is a schematic diagram showing the configuration of a convolutional neural network. FIG. 5 is a diagram illustrating an example of the configuration of a filter. FIG. 6 is a diagram for explaining the filtering process. FIG. 7 is a diagram showing an example in which max pooling is applied to a pooling layer. FIG. 8 is a diagram for explaining the function of the first dropout layer. FIG. 9 is a schematic diagram showing the configuration of a modified example of the convolutional neural network shown in FIG. FIG. 10 is a diagram for explaining the average pooling process. FIG. 11 is a schematic diagram showing the configuration of a steel sheet meandering control system according to one embodiment of the present invention. FIG. 12 is a diagram showing the estimation error of the amount of meandering in the example of the present invention and the comparative example.

One embodiment of the present invention will be described below with reference to the drawings.

[Continuous steel plate conveying equipment]
First, with reference to FIG. 1, the configuration of a continuous conveying facility for steel sheets according to an embodiment of the present invention will be described.

Figure 1 is a schematic diagram showing the configuration of a continuous conveying equipment for steel sheets according to one embodiment of the present invention. As shown in the figure, a continuous conveying equipment for steel sheets (hereinafter, abbreviated as continuous conveying equipment) 1 according to one embodiment of the present invention is equipment that continuously conveys steel sheets S from the left side to the right side of the figure. The steel sheets S pass through a processing equipment 2 that performs various processes on the steel sheets S while being continuously conveyed by the continuous conveying equipment 1. The processing equipment 2 is, for example, a heat treatment furnace in a continuous annealing line, and may be a part of a heat treatment furnace that is divided into multiple sections, such as a preheating zone, a heating zone, a soaking zone, and a cooling zone. For example, when the processing equipment 2 is a cooling zone of a continuous annealing line, the steel sheets S to be heat treated are charged into the processing equipment 2 in a state where they are soaked to a predetermined temperature (about 600 to 900 ° C.). The steel sheets S charged into the processing equipment 2 are then cooled to a predetermined temperature by multiple conveying rolls 3 before being carried out from the exit of the processing equipment 2.

Here, examples of cooling means for the steel sheet S include gas jet cooling, roll cooling, water cooling (water quenching), etc. Gas jet cooling is a cooling means in which gas is sprayed from a nozzle onto the surface of the steel sheet S. Roll cooling is a cooling means in which the steel sheet S is cooled by contacting it with a water-cooled roll. Water cooling is a cooling means in which the steel sheet S is cooled by immersing it in a water-cooled tank (not shown) installed in a part of the processing equipment 2. If the temperature of the steel sheet S changes due to these cooling means, the rigidity (longitudinal elastic modulus) of the steel sheet S may change, causing the meandering phenomenon. In addition, the temperature of the steel sheet S may change every time it passes through the transport rolls 3 of the processing equipment 2, making the sheet passing property unstable, causing the meandering phenomenon. In addition, stress and distortion may occur inside the steel sheet S due to thermal contraction, phase transformation, etc., as the temperature of the steel sheet S changes, causing the flatness of the steel sheet S to be disturbed, causing the meandering phenomenon.

[Data Acquisition Section]
FIG. 1 shows an embodiment in which the meandering amount of the steel sheet S is measured on the upstream side of the processing equipment 2 when the steel sheet S is continuously transported. As shown in FIG. 1, the processing equipment 2 is provided with a camera 4 that takes an image of the moving steel sheet S to acquire image data of the steel sheet S, and a meandering amount measuring device (meander meter) 5 that can measure the meandering amount of the steel sheet S. The camera 4 and the meandering amount measuring device 5 may be disposed at any position in the processing equipment 2. Moreover, the camera 4 and the meandering amount measuring device 5 do not necessarily have to be disposed in the processing equipment 2, such as the upstream side of the inlet part of the processing equipment 2 or the downstream side of the outlet part. In this embodiment, the camera 4 and the meandering amount measuring device 5 are disposed at approximately the same position. Approximately the same position means that the camera 4 and the meandering amount measuring device 5 are disposed between two adjacent transport rolls. This is because the meandering amount of the steel sheet S does not change significantly between the position where the camera 4 acquires image data of the steel sheet S and the position where the meandering amount measuring device 5 measures the meandering amount of the steel sheet S, as long as it is between consecutive transport rolls.

When the camera 4 is installed between two adjacent transport rolls, the meandering amount measuring device 5 should be located at a position where the steel sheet S wraps around one of the transport rolls. On the other hand, when the camera 4 is installed at a position where the steel sheet S wraps around a transport roll, the meandering amount measuring device 5 should be located within 5 m of the transport roll. This is because the width of the steel sheet S is generally in the range of about 500 to 2500 mm, and it is considered that the meandering amount of the steel sheet S does not change significantly up to a position about 2 to 10 times the width of the steel sheet S to be measured. However, it is more preferable to measure the meandering amount using the meandering amount measuring device 5 at the same position where the camera 4 acquires image data of the steel sheet S.

In this embodiment, a two-dimensional camera (area camera) is used as the camera 4. However, the camera 4 may be a color camera or a monochrome camera. When the processing equipment 2 is a heat treatment furnace or the like, the camera 4 does not necessarily need to be placed inside the processing equipment 2. A window for acquiring image data may be provided in the processing equipment 2, and image data of the steel sheet S may be acquired through, for example, glass. The camera 4 may also be an infrared camera that selectively captures infrared wavelength components. An example of image data of the steel sheet S acquired when a two-dimensional camera is used as the camera 4 is shown in FIG. 2(a). The image data of the steel sheet S shown in FIG. 2(a) is image data of the steel sheet S acquired by a color camera, and is an image of the steel sheet S taken from the position of the camera 4A shown in FIG. 2(c) (hereinafter referred to as image A).

Image A is an example taken from the bottom of the processing equipment 2 toward the downstream side of the processing equipment 2, and shows the steel sheet S wrapped around multiple transport rolls. The three dashed lines attached to image A in FIG. 2(a) represent the positions of the lower part of the transport rolls. Also, FIG. 2(a) shows the position of the edge part when the steel sheet S is wrapped around the most upstream (nearest) transport roll. When the meandering amount of the steel sheet S is measured at the position of the most upstream transport roll using the meandering amount measuring device 5, the feature amount of the most upstream edge part seen above image A is extracted by learning using a convolutional neural network described later, and is associated with the measured meandering amount. This is because the image of the edge part at the position where the steel sheet S is wrapped around the most upstream transport roll in image A is the largest. Therefore, even if an image of the steel sheet S transported by the downstream transport roll 3 is included, it does not become a major obstacle to extracting the position information of the most upstream edge part.

On the other hand, FIG. 2(b) is a schematic diagram showing an image of the steel sheet S (hereinafter referred to as image B) taken by the camera 4B shown in FIG. 2(c). When an image of the steel sheet S is taken at an intermediate position between the transport roll and the next transport roll, a relatively simple image like image B is obtained. Even from such an image, position information on the edge of the steel sheet S can be extracted, and the amount of meandering of the steel sheet S can be estimated. It is preferable that the image data of the steel sheet S includes both ends in the width direction of the steel sheet S. In addition, it is preferable that the imaging range in the longitudinal direction of the steel sheet S includes a length in the transport direction of 0.1 to 10.0 W with respect to the plate width W of the steel sheet S. This is because if the imaging range in the transport direction is shorter than 0.1 W, the position information of the edge portion obtained over the transport direction of the steel sheet S will be reduced. In addition, if the imaging range in the transport direction exceeds 10.0 W, the amount of meandering of the steel sheet S changes within the range of one piece of image data, making it difficult to accurately recognize the position of the edge portion. More preferably, it is in the range of 0.15 to 5.0 W.

In this embodiment, the image data of the steel sheet S includes both ends of the steel sheet S in the width direction, and the amount of meandering does not change much in the transport direction within one image data. In addition to the steel sheet S, the image data of the steel sheet S in this embodiment may also include structures whose positions are fixed within the processing equipment 2, such as the transport rolls 3. This is because the accuracy of estimating the amount of meandering is improved based on the relative positional relationship between the captured structure and the steel sheet S. In addition, it is preferable that the multiple image data acquired by the camera 4 are captured from a fixed position. This is because it is easier to identify the meandering state of the steel sheet S based on the position of the captured image of the steel sheet S.

Return to FIG. 1. As the meandering amount measuring device 5, a device that detects the edge position of the steel sheet S using a backlight method, a radar method, a method of processing camera images (ITV method), an electromagnetic wave type meandering amount measuring device, and other conventional technologies that have been used to measure the meandering amount can be used. Here, the electromagnetic wave type meandering amount measuring device is a device that emits electromagnetic waves toward the edge of the steel sheet S, detects the edge position of the steel sheet S from the propagation time of the reflected wave, and converts this into the meandering amount. By selecting conditions with at least little disturbance for the measurement environment, accurate measurement data on the meandering amount of the steel sheet S can be obtained. In addition, the meandering amount measuring device 5 does not need to be used constantly during the operation of the processing equipment 2, and may be a temporary measuring device. In the processing equipment 2 such as a heat treatment furnace for the steel sheet S, the environment such as heat and atmosphere may have a negative effect on the sensor, so that it may not be possible to use it for a long period of time, but if the purpose is to use it temporarily to collect learning data, a meandering meter with sufficient accuracy and durability can be selected.

The data acquisition unit 10 associates, as a data set, the image data of the steel plate S acquired by photographing the steel plate S with the camera 4 and the measurement data of the meandering amount of the steel plate S acquired by the meandering amount measurement device 5 in synchronization with the timing of acquiring the image data of the steel plate S. However, for the same steel plate S, it is not necessary to completely match the timing of photographing the steel plate S with the camera 4 and the timing of measuring the meandering amount of the steel plate S with the meandering amount measurement device 5. For example, in the example shown in FIG. 1, the camera 4 is disposed at a position upstream of the meandering amount measurement device 5 with respect to the conveying direction of the steel plate S. In this case, the timing of acquiring the image data of the steel plate S with the camera 4 and the timing of acquiring the meandering amount measurement data with the meandering amount measurement device 5 may be simultaneous, but the timing of acquiring the meandering amount measurement data with the meandering amount measurement device 5 may be delayed by the time it takes for the steel plate S to be conveyed from the position of the camera 4 to the position of the meandering amount measurement device 5. In other words, within the range in which the meandering state of the steel sheet S is assumed not to change, there may be a lag between the timing at which the camera 4 acquires image data of the steel sheet S and the timing at which the meandering amount measurement device 5 acquires measurement data on the meandering amount.

The image data acquired by the camera 4 and the measurement data of the meandering amount acquired by the meandering amount measuring device 5 can be a data set acquired at multiple positions in the longitudinal direction of the steel plate S. This is because the meandering amount of the steel plate S may change in the longitudinal direction of the steel plate S. Furthermore, learning data may be acquired at a specific position in the longitudinal direction of the steel plate S, such as at the joint with another steel plate S, at the leading or trailing end of the steel plate S. Furthermore, learning data may be acquired at a constant period (e.g., 1 to 10 sec) for the entire length from the leading end to the trailing end of the steel plate S. The image data of the steel plate S acquired in the above manner and the measurement data of the meandering amount associated with the image data are sent to the meandering amount estimation model generation unit 20 (see FIG. 3).

As shown in FIG. 3, the data acquisition unit 10 may include a screening processing unit 11 for removing a data set with a large error as the measurement data of the meandering amount of the steel sheet S. The screening processing unit 11 judges whether the measurement data of the meandering amount of the steel sheet S is affected by disturbances to the shooting environment. If the disturbances have a large effect on the measurement data of the meandering amount, the screening processing unit 11 excludes the measurement data of the meandering amount and the data set of the image data of the steel sheet S from the learning data of the meandering amount estimation model generation unit 20. Here, disturbances to the shooting environment refer to things that become noise in the image data of the steel sheet S temporarily or over time, such as color tone fluctuations such as color unevenness of the steel sheet S, mist-like water droplets present around the steel sheet S, dirt on the lens of the camera 4 used for shooting, and changes in the background color of the image. For example, if the fluctuation range of the measurement data of the meandering amount continuously acquired in the longitudinal direction of the steel sheet S is greater than a preset threshold value, the screening processing unit 11 removes the image data of the steel sheet S corresponding to the measurement data of the meandering amount. Furthermore, if the fluctuation range of the brightness value of the image data acquired by the camera 4 in the longitudinal direction of the steel sheet S is greater than a preset threshold value, the screening processing unit 11 may remove the image data of the steel sheet S that corresponds to the measurement data of the meandering amount.

[Meander amount estimation model generation unit]
As shown in FIG. 3, the meandering amount estimation model generation unit 20 acquires image data of the steel sheet S acquired by the data acquisition unit 10 and subjected to screening processing as necessary, and measurement data of the meandering amount of the steel sheet S corresponding to the image data, and stores them as a data set in the database unit 21 by associating them. In the meandering amount estimation model generation unit 20, the machine learning unit 23 generates a meandering amount estimation model M that estimates the meandering amount of the steel sheet S using the learning data accumulated in the database unit 21. The meandering amount estimation model generation unit 20 may be located inside a control computer that controls the manufacturing process of the steel sheet S by the continuous conveying equipment 1, or may be configured by hardware separate from the control computer. It may also be arranged in a meandering amount estimation unit 30 (see FIG. 11) described later.

The number of data sets stored in the database unit 21 is 5,000 or more, preferably 10,000 or more, and more preferably 50,000 or more. The data sets may be divided based on attribute information such as the width of the steel plate S, and the meandering amount estimation model M may be generated according to the division. The data sets stored in the database unit 21 may be updated appropriately within a certain number of data sets as an upper limit. The meandering amount estimation model generation unit 20 includes a preliminary processing unit 22 that performs preliminary processing on the data sets stored in the database unit 21 as necessary after the learning data is accumulated in the database unit 21 and before machine learning is performed to generate the meandering amount estimation model M. The preliminary processing performed by the preliminary processing unit 22 is, for example, data compression of image data, standardization of image data and measurement data of meandering amount, data shuffling, etc.

(1) Data Compression Although the resolution of cameras has improved significantly in recent years, machine learning using all information contained in the image data of the steel sheet S as input performance data to estimate the amount of meandering of the steel sheet S may not be efficient in terms of computer memory and calculation time. For this reason, it is advisable to compress the number of pixels in both the horizontal direction (x direction) and vertical direction (y direction) of the two-dimensional image in advance. Specifically, it is preferable to compress the number of pixels of the image data of the steel sheet S to about 50 to 100 in both the x direction and the y direction.

(2) Normalization The preliminary processing unit 22 may normalize the image data, which is the input performance data stored in the database unit 21, and the measurement data of the meandering amount of the steel sheet S, which is the output performance data. Normalization is performed to efficiently update the weight coefficients during machine learning, and is effective in avoiding the gradient vanishing problem during machine learning. When normalizing the input data and the output data, it is preferable to identify the minimum and maximum values of each data and normalize them in the range of 0 to 1. In addition, the average value of the variables may be calculated and normalized in the range of -1 to +1.

(3) Data Shuffling Data sets made up of image data, which is input actual data, and measurement data of the amount of meandering, which is output actual data, are stored in the database unit 21 in the order in which the data was acquired. In response to this, data shuffling may be performed to randomly rearrange the order of the data sets stored in the database unit 21. By changing the order of the data sets acquired in chronological order, the generalization performance of the meandering amount estimation model M can be improved and overlearning can be suppressed.

(4) Data division The data set stored in the database unit 21 may be divided into training data and test data. The training data is used for machine learning, making up 70 to 80% of the entire data set. On the other hand, the remaining data set is used to check whether the meandering amount estimation model M obtained after the machine learning unit 23 completes machine learning can also perform well on the test data.

The machine learning unit 23 uses the data set accumulated in the database unit 21 to generate a meandering amount estimation model M that estimates the meandering amount from the image data of the steel sheet S by machine learning using multiple learning data, in which the image data is used as input actual data and the measurement data of the meandering amount of the steel sheet S is used as output actual data. The machine learning model for generating the meandering amount estimation model M may be any machine learning model as long as it can obtain sufficient estimation accuracy for the meandering amount for practical use. For example, commonly used neural networks, decision tree learning, random forests, support vector regression, etc. may be used. Also, an ensemble model combining multiple models may be used. However, this embodiment is characterized in that a convolutional neural network (CNN) is used as a machine learning method.

A convolutional neural network is a neural network that includes a convolutional layer and a pooling layer, and has excellent image discrimination capabilities. Here, the configuration of the convolutional neural network of this embodiment will be described with reference to FIG. 4. When the image data of the steel sheet S is color, the image data can convert the brightness value for each channel into numerical information, for example, 0 to 255, corresponding to the three RGB channels. In other words, the image information obtained from the color image data is information of the number of vertical pixels of the image x the number of horizontal pixels x brightness value x number of channels. In the example shown in FIG. 4, array information of the number of vertical pixels x the number of horizontal pixels x brightness value is described for each channel.

As shown in FIG. 4, the convolutional neural network of this embodiment includes a first convolutional layer, a second convolutional layer, a pooling layer, a fully connected layer, and an output layer in this order for these image data (input data). In addition, a first dropout layer and a second dropout layer are provided upstream and downstream of the fully connected layer, respectively. In this convolutional neural network, the first convolutional layer and the second convolutional layer perform a filtering process on the input data using a filter called a kernel to generate a feature map. Convolution refers to a calculation process in which a filter is applied to the input data to generate an output called a feature map. The filter used for the filtering process is, for example, a filter of 3 pixels vertically by 3 pixels horizontally as shown in FIG. 5. A weighting factor ωij (i indicates the row number in the filter and j indicates the column number in the filter) is assigned to the nine pixel positions in this filter, and the product of the brightness value and the weighting factor at each pixel position of the input image to be filtered is calculated, and the sum of these is calculated. A bias is provided for the calculated sum in the filter, and an output value is calculated using an activation function, and this value is assigned to the pixel position of the feature map. The filter size value corresponds to the number of pixels in the vertical and horizontal directions of the image. For example, a filter size of 3 refers to a filter of 3 pixels vertically by 3 pixels horizontally, as shown in FIG. 5. In the convolution layer of this embodiment, it is preferable to use a nonlinear function as the activation function, and it is preferable to use the Relu function so that the gradient vanishing problem during learning can be suppressed.

As shown in Figs. 6(a) to (f), the above filters generate a feature map corresponding to one input image by calculating the output value at each position while moving the filter position vertically and horizontally within the image. The amount of movement when moving the filter position is called the stride, and the size of the first feature map changes depending on the number of strides set. A stride of 1 means that the filter position is moved by one pixel at a time. Multiple filters can be used to generate the feature map, and discrimination performance is improved by combining multiple filters. In this way, multiple feature maps corresponding to the number of filters used for one input image are generated. When generating the feature map, padding may be performed to fill the periphery of the input image with zeros. This is because it is possible to prevent a lack of information on the edges of the original image.

In a convolutional neural network, this convolutional layer makes it possible to identify the edge position of the steel plate S in the image data. Furthermore, if a fixed structure is included in the image of the steel plate S acquired by the camera 4, image features reflecting the relative positional relationship between the steel plate S and the fixed structure in the image data are extracted, so that the amount of meandering of the steel plate S can be estimated even if the multiple image data acquired by the camera 4 are images taken from different positions.

In this embodiment, a first feature map is generated by the first convolution layer for image data that is input to the convolutional neural network, and a second feature map is generated by the second convolution layer. Since the image data in this embodiment is targeted at planar and relatively simple images such as steel plate S, by providing two convolution layers, it is possible to efficiently extract the features of the image and suppress overlearning. Here, a preferred example of the first convolution layer that generates the first feature map is a convolution layer with 32 to 128 filters, 3 to 5 filter sizes, and 1 stride. Also, a preferred example of the second convolution layer that generates the second feature map is a convolution layer with 64 to 256 filters, 3 to 5 filter sizes, and 1 stride.

As shown in FIG. 2(b), the image data of the steel plate S acquired in this embodiment is preferably captured so that the longitudinal direction of the steel plate S is vertical, and the left and right edge portions of the steel plate S are captured so that they are in the vertical direction of the image. In this case, the left and right edge portions of the steel plate are recognized as straight lines in one image. Therefore, even if the input image data is subjected to convolution processing only in the horizontal direction, it is considered possible to identify the edge position of the steel plate S in the image. However, in this embodiment, the input image data is subjected to convolution processing in the horizontal and vertical directions. By performing convolution processing in the vertical direction as well, even if noise is contained in a part of the image, the average position information of the edge position is identified from the vertical direction in the image, so that stable edge position identification is possible. In particular, there are cases in which disturbances to the shooting environment such as water vapor are contained in the continuous conveying equipment 1 for steel plates, and two-dimensional convolution processing is effective. The same effect can be obtained even if the image data of the steel plate S captured in this invention is captured so that the longitudinal direction of the steel plate S is horizontal, and the left and right edge portions of the steel plate are captured so that they are in the vertical direction of the image. Also, as in the example shown in FIG. 2(a), the longitudinal direction of the steel plate may be photographed vertically, and the image of the steel plate for determining the amount of meandering may be photographed only partially within the entire image.

The information of the second feature map generated in the second convolution layer is aggregated in the pooling layer to generate a third feature map. The pooling layer has the role of compressing the information of the second feature map. FIG. 7 shows an example in which max pooling is applied to the pooling layer (an example of stride 2). Max pooling is a process in which the second feature map, which is the input of the pooling layer, is divided into a certain area (pool size) (3 rows x 3 columns in the example of FIG. 7), the maximum value is extracted from the area, and output as a new feature map. However, the filter used in the pooling layer may not extract the maximum value, but may extract the average value. Such a pooling layer can reduce the amount of information while maintaining the features of the input image, and generate a dimensionally compressed third feature map. The pooling layer has a function of compressing information, so that even if the input image contains disturbances to the shooting environment, the feature amount related to the edge position of the steel sheet S can be detected. A plurality of sizes of filters can be applied to the pooling layer, and in this case, a number of third feature maps corresponding to the type of filter are generated. An example of a preferred form of the pooling layer that generates the third feature map is max pooling with a stride of 2 and a pooling size of 3 (3 rows x 3 columns). However, since the pooling layer aims to reduce the size of the output data while retaining the features of the image, padding that fills the periphery of the output data with 0s is not performed.

Return to FIG. 4. In the convolutional neural network of this embodiment, a fully connected layer for converting the third feature map generated in the pooling layer and an output layer are connected. The fully connected layer is arranged to arrange the values of the third feature map in a row and to consolidate the output from the pooling layer. The structure of the fully connected layer is similar to that of the intermediate layer of a normal neural network. An example of a preferred form of the fully connected layer is a fully connected layer with 512 to 2048 nodes. The output layer outputs the position information of the edge part of the steel sheet S in the image, or the meandering amount of the steel sheet S. The meandering amount of the steel sheet S can be calculated if the position information of the edge part of the steel sheet S in the image is specified, and the relationship between the position information of the width direction center part of the continuous conveying equipment 1 in the acquired image and the distance between the two points in the image and the distance in real space is known in advance, so either can be used as the output of the meandering amount estimation model M. For example, if a fixed structure of the equipment is photographed in the image of the steel sheet S, the positional relationship between the position of the width direction center part of the continuous conveying equipment 1 and the position of the photographed fixed structure is known in real space. Furthermore, if the width of the steel sheet S is known in advance, the relationship between the distance between two points in the image and the distance in real space is also known. On the other hand, the output layer may use a softmax function to calculate the probability that the image data of the steel sheet S included in the input image matches the corresponding measurement data of the meandering amount, and output information on the meandering amount with the highest probability. Also, information on the meandering amount classified by a classifier using a support vector machine that uses the fully connected layer as input may be output. Note that, since it is necessary to perform regression calculations in the output layer, it is preferable to use an identity function as the activation function used in the fully connected layer.

In this embodiment, the convolutional neural network includes a first dropout layer between the pooling layer and the fully connected layer. Also, a second dropout layer is provided between the fully connected layer and the output layer. The first dropout layer and the second dropout layer do not have to be arranged. The first dropout layer randomly cuts off a part of the connection between the pooling layer and the fully connected layer. Also, the second dropout layer randomly cuts off a part of the connection between the fully connected layer and the output layer. This makes it possible to prevent overlearning. As illustrated in FIG. 8, when a filter (3 rows x 3 columns in the example of FIG. 8) is applied to the third feature map, the first dropout layer cuts off a part of the connection between the pooling layer and the fully connected layer by setting the weight coefficient of a randomly selected part in the filter to zero, and outputs it as a fourth feature map. The selection of the part whose weight coefficient is set to zero is determined based on a pseudorandom number output by a computer. Also, a preferred form of the first dropout layer in this embodiment is a dropout layer with a probability of 0.4. A preferred form of the second dropout layer is a dropout layer with a probability of 0.4. The probability of the dropout layer means the ratio of the number of weight coefficients that are set to zero based on the pseudorandom numbers to the total number of weight coefficients.

When using the above-mentioned convolutional neural network method, the number and combination of convolutional layers and pooling layers used in this embodiment may be selected arbitrarily. In addition, as a network structure, LeNet, AlexNet, VGG (Visual Geometry Group), etc. may be used as a commonly used network. Furthermore, as a more complex network structure, GoogleNet, MobileNet, EfficientNet, etc. may be used. Note that, as an input image in this embodiment, convolution processing may be performed directly on color image data, but the input image may also be converted into one-channel information such as grayscale or R value only, and then the convolution processing may be performed on such an input image.

As a machine learning method in the machine learning unit, the data set stored in the database unit 21 can be divided into training data and test data for learning, thereby improving the accuracy of estimating the meandering amount. For example, the weight coefficients of the convolutional neural network can be learned using the training data, and the meandering amount estimation model M can be obtained while appropriately changing the structure of the convolutional neural network (the number of convolutional layers and pooling layers, the filter size, etc.) so that the accuracy rate of the meandering amount in the test data is high. The error propagation method can be used to update the weight coefficients. Note that the meandering amount estimation model M may be updated to a new model by re-learning, for example, every month or every year. This is because the more data stored in the database unit 21, the more accurate the meandering amount estimation becomes, and by updating the meandering amount estimation model M based on the latest data, a meandering amount estimation model M that reflects changes in operating conditions over time can be generated.

Furthermore, in this embodiment, it is preferable to use an average pooling layer upstream of the first convolution layer and connected to the first convolution layer in the above-mentioned convolution neural network. In the example shown in FIG. 9, the average pooling layer provided upstream of the first convolution layer is called the first pooling layer, and the pooling layer downstream of the second convolution layer and connected to the second convolution layer is called the second pooling layer. In this embodiment, by providing the average pooling layer as the first pooling layer, it is possible to obtain an effect similar to that of converting the input to the convolution layer of the convolution neural network (the input to the first convolution layer) into a "blurred" image. As a result, a state in which there is a disturbance due to the shooting environment around the steel sheet S to be photographed (for example, misty water droplets, etc.) is reproduced to a certain extent, and a convolution neural network with small output fluctuations can be configured even if there is a disturbance (noise) in the shooting environment of the steel sheet S. In addition, since the image of the steel sheet S in this embodiment is a relatively simple image as shown in Figures 2(a) and 2(b), even if an average pooling layer is provided in the first pooling layer, the feature amount related to the position of the edge portion of the steel sheet S (the amount of meandering of the steel sheet) is not lost.

Figure 10 shows an example of average pooling processing. Average pooling processing is processing in which image data that is input data to the first pooling layer is divided into certain regions (3 rows x 3 columns in the example shown in Figure 10), the average value within the regions is extracted, and a new feature map is output. In the preferred form of the first pooling layer in this embodiment, the stride is 1 and the pool size is 3 rows x 3 columns. Furthermore, it is preferable that the number of channels of the first pooling layer is the same as the number of channels of the image data that is input data to the meandering amount estimation model M. For example, when the image data of the steel sheet S is color, the number of channels of the input data is 3 (R, G, B), and the number of channels of the first pooling layer is also 3. On the other hand, when the image data of the steel sheet S is grayscale, the number of channels of the input data is 1, and the number of channels of the first pooling layer is also 1. By performing average pooling processing for each color of the input image data, even if the color tone of the image data of the steel sheet obtained changes due to a change in the temperature of the steel sheet S, a convolutional neural network can be configured in which the position information of the edge part of the steel sheet S that is output (the amount of meandering of the steel sheet) does not fluctuate much.

[Method for estimating the amount of meandering of steel plate]
In this embodiment, the meandering amount of the steel sheet S is estimated using the meandering amount estimation model M generated by the machine learning unit 23. FIG. 11 shows an example of the configuration of a steel sheet meandering control system including a meandering amount estimation unit in a continuous conveying facility. The camera 4 that acquires the image data of the steel sheet S may be the same as or different from the camera used when generating the meandering amount estimation model M. This is because the meandering amount estimation model M can extract features including position information of the steel sheet S in the image data even if the image data from which the learning data is based is not necessarily an image of the steel sheet S captured in the same field of view. In addition, if the fixed structure included in the image data from which the learning data is based is installed in the same position, the relative positional relationship between the steel sheet S and the fixed structure in the image data is extracted as a feature, and the meandering amount of the steel sheet S can be accurately estimated. The camera that acquires the image data of the steel sheet S does not necessarily have the same resolution as the camera used when generating the meandering amount estimation model M. When the meandering amount estimation model generating unit 20 performs data compression as a preliminary process, image data of the steel sheet S may be acquired at a resolution higher than the same resolution as the compressed image data. On the other hand, the installation position of the camera 4 may be set at a position different from that of the camera 4 used when generating the meandering amount estimation model M. However, it is preferable that the distance between the camera 4 and the steel sheet S and the angle at which the camera 4 faces the steel sheet S are the same.

The meander amount measuring device 5, which measures the meander amount that becomes the learning data in the meander amount estimation model generating unit 20 shown in FIG. 11, is a necessary device for acquiring learning data for generating the meander amount estimation model M, but is not essential when the meander amount of the steel sheet S is estimated by the meander amount estimating unit 30 using the camera 4. Therefore, the meander amount measuring device 5 may be a temporary meander meter used only when acquiring learning data for generating the meander amount estimation model M. In continuous conveying equipment such as a heat treatment furnace for steel sheet S, the environment such as heat and atmosphere adversely affect the sensor, so it cannot be used for long-term operation, but if the purpose is to use it temporarily to collect learning data, a meander meter with sufficient accuracy and durability can be selected.

The processing conditions of the steel sheet S in the processing equipment 2 shown in FIG. 11 do not have to be the same as the processing conditions when the image data that is the source of the learning data was acquired. In other words, the heat treatment temperature and cooling conditions of the steel sheet S may differ from those when the learning data was acquired due to differences in the component composition and product strength of the steel sheet S. Even if the learning data is acquired under conditions with little disturbance from mist droplets or fumes in the continuous conveying equipment, by providing a first pooling layer in the convolutional neural network as the meandering amount estimation model M, a convolutional neural network that is resistant to disturbances in the shooting environment can be generated even if the measurement environment, such as the heat treatment temperature and cooling conditions of the steel sheet S, changes.

In addition, the meandering amount estimation model M generated by machine learning outputs an estimated value of the meandering amount without performing complex image processing or the associated calculation processing by inputting image data captured by the camera 4, so that the meandering amount of the steel sheet S can be output immediately even under conditions where the steel sheet S is transported at high speed. The estimated value of the meandering amount is preferably output to the display device 31. By installing the display device 31 in the operation room of the continuous conveying equipment 1, the operator can check the meandering state of the steel sheet S in the processing equipment 2 at any time, so that the operating conditions of the continuous conveying equipment 1 can be corrected so that the meandering amount of the steel sheet S does not increase and cause a conveying trouble. In addition, as shown in FIG. 11, when the processing equipment 2 is equipped with a meandering correction device for the steel sheet S such as a steering roll 6, the meandering amount estimation model M can output the meandering amount of the steel sheet S estimated by the meandering amount estimation unit 30 for each control cycle of, for example, about 100 to 1000 msec. By changing the operation amount of the steering roll 6 according to the meandering amount estimated by the meandering control unit 40, the meandering amount of the steel sheet S during conveying can be effectively reduced. In addition, by reducing the amount of meandering, stable processing of steel sheet S becomes possible, improving the production efficiency of steel sheet S.

Below, an example in which this embodiment is applied to a continuous annealing facility for steel strips will be described. In this example, a meandering amount estimation model was first generated. A color camera capable of capturing an image of a steel sheet moving in the annealing furnace was installed outside the furnace observation window on the inlet side of the continuous annealing furnace. The camera was positioned so that the steel sheet moved from top to bottom in the continuous conveying facility, and the magnification and lens were selected so that both ends of the steel sheet in the width direction were captured. The image data captured by the camera was sent to the data acquisition unit and accumulated as image data of the steel sheet. Here, for the image data acquired by the camera, the width direction of the steel sheet was defined as the x direction, and the longitudinal direction was defined as the y direction. The camera used was a video camera with a frame rate of 30 fps. In addition, the image captured by the camera has 480 pixels in the x direction, 320 pixels in the y direction, and 3 channels.

On the other hand, the measurement data of the meandering amount was acquired by an ITV camera type meandering amount measuring device 5 capable of measuring the meandering amount of the steel sheet from below the conveying rolls about 3 m below the position where the camera 4 takes the image, as shown in FIG. 11. The ITV camera type meandering amount measuring device 5 extracts the edge of the steel sheet S from the image captured by the camera by binarization processing, and measures the meandering amount of the steel sheet S from the correspondence between the position information in the image acquired in advance and the position information of the actual equipment. The ITV camera type requires a certain amount of time for image processing (such as binarization processing and edge extraction processing), and measurement errors may occur due to disturbances in the measurement environment. However, it is possible to accurately determine the meandering amount by recording the images of the ITV camera, selecting images that are judged to have small measurement errors, and performing the above image processing offline.

For the measurement data of the meander amount measured by the meander amount measuring device 5 that was judged to have obtained a correct measurement value, the image data of the steel sheet S acquired in synchronization with the measurement timing was sent as a data set to the database unit 21 of the meander amount estimation model generating unit 20. In addition, in the data acquiring unit 10, if any of the acquired image data and meander amount data contains an abnormal value, a screening process was performed to remove the data set (a set of image data and meander amount data) containing the abnormal value from the learning data. This is because the dew point inside the continuous annealing furnace may be intentionally changed when processing the steel sheet S, and depending on the continuous annealing conditions, the image captured or the measurement data by the meander amount measuring device may be affected by external disturbances. In addition, the steel sheet S to be measured had a plate thickness of 0.4 to 2.4 mm and a plate width of 700 to 1850 mm, and included ordinary steel and high-strength steel plates.

In this manner, the meandering amount estimation model M was generated by machine learning in the meandering amount estimation model generation unit 20 using a data set of image data of the steel plate and corresponding measurement data of the meandering amount. When generating the meandering amount estimation model M, data of low carbon steel with a plate thickness of 1.2 mm and a plate width of 1040 mm was selected from the data set stored in the database unit 21, and the meandering amount estimation model M was generated from the selected data. In the preliminary processing unit 22 of the meandering amount estimation model generation unit 20, data compression was performed on the image data of the steel plate stored in the database unit 21, compressing the number of pixels in the x direction and y direction to 50 each. In addition, after removing outliers from the data set, the data was shuffled and divided into 56,160 learning data and 14,040 test data.

In this embodiment, as Example 1 of the present invention, a meandering amount estimation model M was generated using the configuration of the convolutional neural network shown in FIG. 4. The loss function in the learning calculation was the mean square error calculated from the predicted value of the network and the actual measured value, and Adam optimization was selected as the optimization calculation method. In addition, mini-batch learning (mini-batch gradient descent method) was used for machine learning, with a batch size of 512 and the number of epochs of 500. However, if the loss function for the test data did not improve for 30 epochs or more, the learning rate was reduced to 1/10 and learning was continued. Furthermore, as Example 2 of the present invention, a meandering amount estimation model M was generated using the configuration of the convolutional neural network shown in FIG. 9. This is a model in which a first pooling layer that performs average pooling processing on each of the three RGB channels of the input image is added upstream of the first convolutional layer. The loss function and optimization calculation method in learning are the same as those in Example 1 of the present invention. On the other hand, as a comparative example, a forward propagation type neural network was constructed using the same learning data as in invention examples 1 and 2, which converts the input image into one-dimensional information to estimate the amount of meandering of the steel sheet. The neural network used had three intermediate layers and five nodes each. A sigmoid function was used as the activation function.

Figure 12 shows the evaluation results of the estimation error using test data for Examples 1 and 2 of the present invention and the comparative example. In the comparative example, the standard deviation for the test data was about 40 mm, but in Example 1 of the present invention using a convolutional neural network, it was 18 mm, and in Example 2 of the present invention adding the first pooling layer, it was 4.4 mm. In addition, using the generated meandering amount estimation model, the prediction accuracy for newly acquired test data six months and one year after the trained model was generated was evaluated. As a result, the error standard deviation after six months and one year had deteriorated to 44 mm and 55 mm in the comparative example. In contrast, in Example 1 of the present invention, good accuracy was maintained at 15 mm and 16.2 mm, respectively. In Example 2 of the present invention, it was 4.6 mm and 4.77 mm, and even better accuracy was maintained.

REFERENCE SIGNS LIST 1 Continuous conveying equipment 2 Processing equipment 3 Conveying rolls 4, 4A, 4B Camera 5 Meandering amount measuring device 10 Data acquisition section 11 Screening processing section 20 Meandering amount estimation model generating section 21 Database section 22 Preliminary processing section 23 Machine learning section 30 Meandering amount estimation section 31 Display device 40 Meandering control section M Meandering amount estimation model S Steel plate

Claims (4)

a data acquiring step of acquiring image data of the steel sheet by photographing the moving steel sheet and measurement data of the meandering amount of the steel sheet corresponding to the image data;
a model generation step of generating a meandering amount estimation model for estimating the meandering amount of the steel sheet by machine learning by a convolutional neural network technique using a plurality of learning data, the model generation step using the image data acquired in the data acquisition step as input record data and measurement data of the meandering amount of the steel sheet corresponding to the image data as output record data;
Including ,
The method for generating a model for estimating a meandering amount of a steel sheet , wherein the convolutional neural network includes one or more average pooling layers upstream of a convolutional layer. The method for generating a steel sheet meandering amount estimation model according to claim 1 , wherein the number of channels of the average pooling layer is the same as the number of channels of image data that is input data to the steel sheet meandering amount estimation model. 3. A method for estimating the meandering amount of a steel sheet, comprising: estimating the meandering amount of the steel sheet by inputting image data acquired by photographing a moving steel sheet to a meandering amount estimation model generated by the method for generating a meandering amount estimation model of a steel sheet according to claim 1 or 2. A method for manufacturing a steel sheet, comprising a step of controlling a meandering amount of a steel sheet in a continuous annealing facility by using the method for estimating the meandering amount of a steel sheet according to claim 3 .

Images