JP7343748B2 - Image judgment learning device, image judgment learning program, image judgment learning method, and blast furnace monitoring device - Google Patents

Description

The present invention relates to an image judgment learning device, an image judgment learning program, an image judgment learning method, and a blast furnace monitoring device.

Traditionally, gas flow stability and furnace conditions in blast furnaces have been monitored based on changes in shaft pressure and absolute values of stave temperature. In addition, the reaction status and fluctuation status in the raceway and lower part of the cohesive zone can be monitored by the operator visually observing the brightness inside the blast furnace, the swirling status of coke, the dripping of unreduced ore, etc. through the tuyeres. It had been. However, visual monitoring by an operator may not sufficiently ensure consistency, objectivity, and data storability in monitoring the reaction status of a raceway or the like. For example, in visual monitoring by an operator, the criteria for determining whether an abnormal condition has occurred depends on the operator's subjectivity, and therefore varies depending on the operator performing the monitoring work, and there is a risk that consistency and objectivity may not be maintained. .

In order to maintain consistency in monitoring the reaction status of raceways, etc., technology has been developed to observe raceways, etc. from the tuyere using imaging devices such as raceway depth meters, CCD cameras, and sensors such as radiation thermometers. known (for example, see Patent Documents 1 to 7). For example, the technology described in Patent Document 1 calculates the length of a perpendicular line drawn from the representative brightness vector in the direction of the principal component vector as an evaluation value, and compares the evaluation value with a predetermined threshold to determine whether there is an abnormality in the blast furnace. Determine. By using the techniques described in Patent Documents 1 to 7, the reaction status of the raceway and the lower part of the cohesive zone can be monitored with consistency, objectivity, and data preservation.

Additionally, there is a known technique for estimating the temperature distribution of the combustion field in the raceway using thermal images taken from the tuyere of the combustion field in the raceway at two different wavelengths (see, for example, Patent Documents 8 to 9). . By using the techniques described in US Pat.

JP2015-25188A Patent No. 5999155 Patent No. 5935828 Patent No. 5867619 Japanese Patent Application Publication No. 2015-227478 Japanese Patent Application Publication No. 2015-52149 JP2015-52148A Japanese Patent Application Publication No. 2001-318002 Japanese Patent Application Publication No. 2002-309307

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, "ImageNetClassification with Deep Convolutional Neural Networks", Advances InNeural Information Processing Systems, Vol.25, pp.1106 - 1114, 2012.

Items that the operator visually monitors from the tuyere include the swirling status of coke, the falling of raw ore, which is a phenomenon in which ore that is not completely melted falls, and the presence or absence of damage to the tuyere and damage to the pulverized coal supply pipe. Further examples include the presence or absence of damage to blast furnace equipment, such as the presence or absence of damage. However, since images need to be recognized from a human perspective to determine the coke swirling state, raw ore fall state, and whether there is damage to blast furnace equipment, the techniques described in Patent Documents 1 to 9 cannot be used. It is not easy to monitor.

One embodiment aims to generate a learning device that can be used to monitor operating conditions that require image recognition to be performed from the perspective of a human being, without relying on visual monitoring by an operator.

The gist of the present invention to solve such problems is an image judgment learning device, an image judgment learning program, an image judgment learning method, and a blast furnace monitoring device described below.
(1) A plurality of image data showing a plurality of images taken at different times by an imaging device having an imaging unit placed opposite to any one of the tuyeres that supply pulverized coal together with gas into the blast furnace. an image data acquisition unit that acquires
a clustering unit that clusters images corresponding to each of the plurality of image data into at least three image groups;
a labeling unit that associates each of the at least three image groups with an operational state of the blast furnace;
When it is determined that the operational state of the blast furnace associated with at least three image groups includes a normal state, a first abnormal state, and a second abnormal state, which of the normal state, the first abnormal state, and the second abnormal state a learning data determining unit that determines, as learning data, image data corresponding to a plurality of images clustered into one image group;
a learning processing unit that causes a learning device to learn the relationship between the plurality of images and the operational status of the blast furnace using the learning data;
a learning device output unit that outputs a learning device that has learned the relationship between the plurality of images and the operational status of the blast furnace;
An image judgment learning device characterized by having the following.
(2) When determining that the operational state of the blast furnace associated with at least three image groups does not include at least one of the normal state, the first abnormal state, and the second abnormal state, the clustering unit selects the images to be clustered. The image judgment learning device according to (1), which increases the number of groups and executes the clustering process again.
(3) The first abnormal state is a phenomenon in which ore that is not completely melted falls, and the second abnormal state is a phenomenon in which the pulverized coal supply pipe that supplies pulverized coal to the tuyere is damaged. (1 ) or the image judgment learning device according to (2).
(4) The image data acquisition unit acquires run purge time information indicating the run purge time during which pulverized coal is not supplied from the tuyere to the inside of the blast furnace, and extracts multiple image data by excluding images captured during the run purge time. The image judgment learning device according to any one of (1) to (3).
(5) When there is a group of images that are not associated with any of the normal state, the first abnormal state, and the second abnormal state, the labeling unit is configured to label the image group with the image group that is not associated with any of the normal state, the first abnormal state, and the second abnormal state. The image judgment learning device according to any one of (1) to (4), which is associated with a caution-required state indicating a certain condition.
(6) a learning device that outputs the operating state of the blast furnace according to the image learned and input by the image judgment learning device according to any one of (1) to (5);
a current image data acquisition unit that acquires current image data indicating a current image captured by the imaging device;
a current image input unit that inputs current image data to the learning device;
an operating state acquisition unit that obtains the operating state of the blast furnace according to current image data input from the learning device;
an operating state output unit that outputs an operating state signal indicating the operating state of the blast furnace output from the learning device;
A blast furnace monitoring device characterized by having:
(7) A plurality of image data showing a plurality of images taken at different times by an imaging device having an imaging unit placed opposite to any one of the tuyeres that supply pulverized coal together with gas into the blast furnace. and
Clustering images corresponding to each of the plurality of image data into at least three image groups,
associating each of the at least three image groups with an operational state of the blast furnace;
When it is determined that the operating state of the blast furnace associated with at least three image groups includes a normal state, a first abnormal state, and a second abnormal state, which of the normal state, the first abnormal state, and the second abnormal state image data corresponding to a plurality of images clustered into a group of images associated with one is determined as training data;
Using learning data, the learning device learns the relationship between multiple images and the operational status of the blast furnace,
Outputs a learning device that learns the relationship between multiple images and the operational status of the blast furnace.
An image judgment learning method characterized by including the following.
(8) A plurality of image data showing a plurality of images taken at different times by an imaging device having an imaging unit placed opposite to any one of the tuyeres that supply pulverized coal together with gas into the blast furnace. and
Clustering images corresponding to each of the plurality of image data into at least three image groups,
associating each of the at least three image groups with an operational state of the blast furnace;
When it is determined that the operational state of the blast furnace associated with at least three image groups includes a normal state, a first abnormal state, and a second abnormal state, which of the normal state, the first abnormal state, and the second abnormal state image data corresponding to a plurality of images clustered into a group of images associated with one is determined as training data;
Using learning data, the learning device learns the relationship between multiple images and the operational status of the blast furnace,
Outputs a learning device that learns the relationship between multiple images and the operational status of the blast furnace.
An image judgment learning program characterized by causing a computer to perform processing.

In one embodiment, operating conditions that require image recognition from a human perspective can be monitored without requiring visual monitoring by an operator.

1 is a diagram schematically showing a configuration example of a blast furnace monitoring system including a blast furnace monitoring device according to an embodiment. It is an example of the image imaged by the imaging device shown in FIG. FIG. 1 is a diagram showing an image judgment learning device according to a first embodiment. 4 is a flowchart of image judgment learning processing executed by the image judgment learning device shown in FIG. 3. FIG. 2 is a diagram showing an example of a representative image associated with each of the operational states of the blast furnace shown in FIG. 1, in which (a) is an example of a representative image associated with a run purge state, and (b) is a representative image associated with a normal state. These are examples of images, (c) is an example of a representative image associated with a caution-required state, (d) is an example of a representative image associated with a raw ore drop state, and (e) is an example of a representative image associated with a broken lance state. This is an example of a representative image. 2 is a diagram showing the blast furnace monitoring device shown in FIG. 1. FIG. 7 is a flowchart of an operation state estimation process executed by the blast furnace monitoring device shown in FIG. 6. FIG. 3 is a diagram showing an example of an estimation result by a blast furnace monitoring device equipped with a learning device learned by the image judgment learning device shown in FIG. 2. FIG. FIG. 7 is a diagram showing an image judgment learning device according to a second embodiment. 10 is a flowchart of image judgment learning processing executed by the image judgment learning device shown in FIG. 9. It is a figure showing an image judgment learning device concerning a 3rd embodiment. 12 is a flowchart of image judgment learning processing executed by the image judgment learning device shown in FIG. 11.

An image judgment learning device, an image judgment learning program, an image judgment learning method, and a blast furnace monitoring device will be described below with reference to the drawings. However, the technical scope of the present invention is not limited to these embodiments.

(Overview of image judgment learning device according to embodiment)
The image judgment learning device according to the embodiment clusters images taken of the inside of the blast furnace from the tuyere into at least three image groups, and determines whether any of the clustered image groups indicates that the operating state of the blast furnace is in a normal state or a raw ore falling state. and determine whether or not it indicates damage to the blast furnace equipment. The image determination learning device according to the embodiment transmits data corresponding to each of the image groups when it is determined that any of the image groups indicates that the operational state of the blast furnace is in a normal state, a raw ore dropout state, or damage to the blast furnace equipment. This is used to make the learning device learn the relationship between images and the operational status of the blast furnace. Since the image judgment learning device according to the embodiment makes the learning device learn using a group of images showing the falling raw ore state and damage to blast furnace equipment, the blast furnace monitoring device equipped with this learning device can It is possible to monitor the state of fallen ore and the presence or absence of damage to blast furnace equipment.

(Blast furnace monitoring system according to embodiment)
FIG. 1 is a diagram schematically showing a configuration example of a blast furnace monitoring system including a blast furnace monitoring device according to an embodiment.

In the blast furnace 100, about 40 tuyeres 102 are arranged around the furnace body 101. A hot air supply pipe 103, a pulverized coal supply pipe 104, an observation window 105, and an imaging device 106 are arranged in each of the tuyeres 102, and pulverized coal is supplied to the inside of the blast furnace 100 together with high pressure gas.

The hot air supply pipe 103 is a cylindrical member inserted into the furnace body 101 with one end having a tuyere 102, and high-temperature gas is blown into the blast furnace 100 at high pressure from the other end. A raceway 107 is formed inside the blast furnace 100 by the wind pressure of the high temperature gas blown into the inside of the blast furnace 100 through the hot air supply pipe 103 .

The pulverized coal supply pipe 104 is a cylindrical member with one end inserted into the hot air supply pipe 103, and pulverized coal is blown into the blast furnace 100 from the other end. In one example, a pair of pulverized coal supply pipes 104 are arranged in each of the tuyeres 102 . At the interface of the raceway 107, a high-temperature combustion reaction occurs in which the coke charged into the blast furnace 100 and the pulverized coal injected from the pulverized coal supply pipe 104 are burned to generate carbon monoxide. In order to prevent the inside of the pulverized coal supply pipe 104 from being clogged with pulverized coal, the injection of pulverized coal is stopped at predetermined time intervals, and high-pressure gas such as nitrogen is blown into the pulverized coal supply pipe 104 instead of the pulverized coal. An operating state in which high pressure gas is blown in instead of pulverized coal is also called a run purge state. Further, even when the pulverized coal supply pipe 104 is clogged, high pressure gas such as nitrogen is blown into the pulverized coal supply pipe 104 instead of pulverized coal, and the operating state of the blast furnace 100 becomes a run purge state.

The observation window 105 is a light-transmitting member fitted into a hole formed at the other end of the hot air supply pipe 103, and is arranged at a position where the raceway 107 can be visually recognized.

The imaging device 106 is fixedly arranged near the tuyere 102. The imaging device 106 has an imaging unit 108 disposed opposite to the tuyere 102, and captures a plurality of images via the hot air supply pipe 103 and the pulverized coal supply pipe 104 disposed in the tuyere 102, and transmits them to the LAN 109. It is transmitted to the image judgment learning device 1 and the blast furnace monitoring device 110 via. The imaging device 106 is capable of high-speed imaging operation, for example every 10 ms, and sequentially transmits image data indicating captured images to the image judgment learning device 1 and the blast furnace monitoring device 110.

FIG. 2 is an example of an image captured by the imaging device 106. In FIG. 2, a pair of pulverized coal supply pipes 104 are imaged on both left and right sides, and pulverized coal supplied to the raceway 107 via the pulverized coal supply pipes 104 is imaged in the center. In the lower right corner, coke rotating inside the raceway 107 is also imaged.

(Image judgment learning device according to the first embodiment)
FIG. 3 is a diagram showing the image judgment learning device 1.

The image judgment learning device 1 includes a communication section 11, a storage section 12, an input section 13, an output section 14, a processing section 20, and a learning device 30. The communication section 11, the storage section 12, the input section 13, the output section 14, the processing section 20, and the learning device 30 are connected to each other via a bus 200. The image judgment learning device 1 uses image data sent from the imaging devices 106 placed in each of the approximately 40 tuyeres 102 to learn the relationship between the sent images and the operational status of the blast furnace 100. The device 30 is made to learn. In one example, the image judgment learning device 1 may be integrated with a blast furnace monitoring device 110 that monitors the operating state of the blast furnace 100, and in another example, the image judgment learning device 1 may be a personal computer.

The communication unit 11 has a wired communication interface circuit such as Ethernet (registered trademark). The communication unit 11 communicates with the imaging device 106, the blast furnace monitoring device 110, and a higher-level control device (not shown) via the LAN 109.

The storage unit 12 includes, for example, at least one of a semiconductor storage device, a magnetic tape device, a magnetic disk device, or an optical disk device. The storage unit 12 stores operating system programs, driver programs, application programs, data, etc. used in processing by the processing unit 20. For example, the storage unit 12 uses the image transmitted from the imaging device 106 to cause the processing unit 20 to execute an image judgment learning process that causes the learning device 30 to learn the relationship between the image and the operating state of the blast furnace 100. The image judgment learning program and the like are stored. The image judgment learning program may be installed in the storage unit 12 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like. Furthermore, the storage unit 12 stores various data used in the image judgment learning process. For example, the storage unit 12 stores a plurality of image data sequentially transmitted from the imaging device 106 in association with an imaging time.

The input unit 13 may be any device that can input data, such as a touch panel or a keyboard. The operator can use the input unit 13 to input characters, numbers, symbols, and the like. When the input unit 13 is operated by an operator, it generates a signal corresponding to the operation. The generated signal is then supplied to the processing unit 20 as an operator's instruction.

The output unit 14 may be any device that can display videos, images, etc., such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The output unit 14 displays a video according to the video data supplied from the processing unit 20, an image according to the image data, and the like. Further, the output unit 14 may be an output device that prints images, images, characters, etc. on a display medium such as paper.

The processing unit 20 includes one or more processors and their peripheral circuits. The processing unit 20 centrally controls the overall operation of the image judgment learning device 1, and is, for example, a CPU. The processing unit 20 executes processing based on programs (driver programs, operating system programs, application programs, etc.) stored in the storage unit 12. Further, the processing unit 20 can execute multiple programs (application programs, etc.) in parallel.

The processing section 20 includes an image data acquisition section 21 , a clustering section 22 , a labeling section 23 , a learning data determining section 24 , a learning processing section 25 , and a learning device output section 26 . Each of these units is a functional module realized by a program executed by a processor included in the processing unit 20. Alternatively, each of these parts may be implemented in the image judgment learning device 1 as firmware.

The learning device 30 is a learning device that learns the features of an abstracted image through supervised learning. Learn the relationship between the operating status of Deep learning is machine learning using a multilayer neural network consisting of an input layer, a hidden layer, and an output layer. Each node of the input layer receives a feature vector of each of the plurality of images captured by the imaging device 106. The feature vector may be, for example, a partial image obtained by cutting out a part of the image. Further, each of the partial images used as a feature vector may be cut out so that a portion thereof overlaps with another partial image. Each node in the middle layer outputs the sum of the values obtained by multiplying each feature vector output from each node in the input layer by a weight, and the output layer outputs the sum of the values obtained by multiplying each feature vector output from each node in the middle layer by a weight. Outputs the sum of the values multiplied by the weights. The learning device 30 learns while adjusting each weight so that the difference between the output value from the output layer and each of the plurality of images captured by the imaging device 106 becomes small. The learning device 30 is realized by, for example, the technique described in Non-Patent Document 1.

(Image judgment learning process by image judgment learning device 1)
FIG. 4 is a flowchart of image judgment learning processing executed by the image judgment learning device 1. The image judgment learning process shown in FIG. 4 is mainly executed by the processing unit 20 in cooperation with each element of the image judgment learning device 1 based on a program stored in the storage unit 12 in advance. In addition, since the states of the raceway 107 that are visually recognized from about 40 tuyeres 102 are different from each other, the image judgment learning process uses image data transmitted from the imaging device 106 arranged at each of the tuyeres 102. and is executed for each of the tuyeres 102.

First, the image data acquisition unit 21 acquires a plurality of image data representing a plurality of images captured at different times by the imaging device 106 corresponding to the target tuyere 102 (S101). The image corresponding to the image data acquired by the image data acquisition unit 21 may be an image captured every 10 ms over one month, for example.

Next, the clustering unit 22 clusters the images corresponding to the image data acquired by the image data acquisition unit 21 into five image groups (S102). Specifically, the clustering unit 22 clusters the images corresponding to the image data acquired by the image data acquisition unit 21 into five image groups using a known clustering method such as the k-means method.

Next, the labeling unit 23 associates each of the five image groups clustered by the clustering unit 22 with the operating state of the blast furnace 100 (S103). Specifically, the labeling unit 23 uses representative images associated with each operating state of the blast furnace 100 to label five images based on the similarity (distance) between each of the five image groups and the representative image. Each of the groups is associated with an operating state of the blast furnace 100. That is, the operating state of a representative image whose degree of similarity with the image group is a predetermined value or more (distance is a predetermined value or less) is associated with the image group. If there are multiple representative images whose degree of similarity to the image group is greater than or equal to a predetermined value (distance is less than or equal to a predetermined value), the operating state of the representative image with the maximum degree of similarity (distance is the minimum) is associated with the image group. Good too. Note that the degree of similarity between the image group and the representative image can be the degree of similarity between the image located at the center (center of gravity) of the image group and the representative image.

Here, the representative image is a tuyere image that typically represents a predetermined operating state of the blast furnace 100, and is prepared in advance. For example, based on a record of the operating state of the blast furnace 100 stored in a higher-level control device (not shown), a tuyere image acquired at a certain timing in the past when a predetermined operating state was recorded is used as a representative image. or may be selected by an operator skilled in monitoring the blast furnace 100. Furthermore, when the learning device 30 performs learning multiple times in each predetermined learning cycle, the representative image used in the previous learning cycle is stored, so the same representative image can be used. .

FIG. 5 is a diagram showing an example of representative images associated with each operating state of the blast furnace 100.

FIG. 5(a) is an example of a representative image associated with the run purge state. In the run purge state, high pressure gas such as nitrogen is blown in instead of pulverized coal. Therefore, in the representative image shown in FIG. It has not been.

FIG. 5B is an example of a representative image associated with a normal state in which the internal state of the blast furnace 100 as viewed from the tuyere 102 is good. In the representative image shown in FIG. 5(b), the area other than the path of the pulverized coal injected from the pair of pulverized coal supply pipes 104 is a region of high brightness, indicating that a good high-temperature combustion reaction is occurring in the raceway 107. Show that.

FIG. 5C is an example of a representative image in which the internal state of the blast furnace 100 as viewed from the tuyere 102 is associated with a caution-required state. The caution-required state is, for example, a state immediately before the time when a raw ore drop state or damage to blast furnace equipment occurs.

FIG. 5(d) is an example of a representative image associated with the raw ore drop state. The raw ore drop state is also referred to as the first abnormal state. In the representative image shown in FIG. 5(d), the area from the center of the image to the bottom is a region of low brightness, indicating that ore that is not completely melted is dripping. That is, when unreduced ore comes into contact with coke, it undergoes a direct reduction reaction, which is an endothermic reaction that absorbs a large amount of heat, so the temperature around the ore decreases, which is shown as a dark area on the screen.

FIG. 5E is an example of a representative image associated with a lance broken state in which the pulverized coal supply pipe 104 is broken. The lance broken state is also referred to as a second abnormal state. The lance broken state is a state where the pulverized coal supply pipe 104 is bent or broken for some reason. In the representative image shown in FIG. 5(e), one of the paths of the pulverized coal injected from the pair of pulverized coal supply pipes 104 is curved, indicating that one of the pair of pulverized coal supply pipes 104 is damaged. .

Similar to the clustering unit 22, the labeling unit 23 clusters the images shown in FIG. 5 associated with each state based on the degree of similarity (distance) with the image group. associate with one.

Next, the learning data determination unit 24 determines that the operating states of the blast furnace 100 associated with the five image groups clustered in the process of S102 are a lance purge state, a normal state, a caution required state, a raw ore drop state, and a lance broken state. It is determined whether all of the above are included (S104). When cluster-linking images, if the clusters of images randomly assigned using the k-means method are inappropriate, the images may not be clustered in a state corresponding to each of the representative images used by the labeling unit 23. There is. When images are not clustered in a state corresponding to each of the representative images, that is, when there is no representative image for a certain image group whose similarity with the image group is greater than or equal to a predetermined value (distance is less than or equal to a predetermined value), the labeling unit 23 puts the image group in the "not applicable" state. When the labeling unit 23 marks any of the five image groups as "not applicable", the operating states associated with the five image groups are lance purge state, normal state, caution required state, raw ore drop state, and lance broken. Does not include any of the conditions.

When it is determined that the operating state associated with the five image groups does not include any of the five states such as the run purge state (S104-NO), the learning data determining unit 24 increases the clustering number N by one. and set to 6 (S105).

Thereafter, the processes of S102 to S105 are repeated while increasing the clustering number N by one until it is determined that the operating state associated with the N image groups includes all five states (S104-YES). .

When the learning data determining unit 24 determines that the operating state associated with the N image groups includes all five states (S104-YES), the learning data determination unit 24 selects the plurality of images clustered into the image group in the process of S102. The image data corresponding to is determined as learning data (S106).

Next, the learning processing unit 25 causes the learning device 30 to learn the relationship between the plurality of images and the operating state of the blast furnace 100 using the learning data determined in the process of S106 (S107). For example, the learning processing unit 25 may input a partial image obtained by cutting out a partial region of a plurality of images to the input layer of the learning device 30 as a feature vector. Each of the partial images input to the input layer may be cut out so that a portion thereof overlaps with another partial image. Further, the learning processing unit 25 may perform image processing such as adjusting the contrast of each of the plurality of images before inputting the partial images to the input layer of the learning device 30.

Then, the learning device output unit 26 outputs the learning device 30 that has learned the relationship between the plurality of images captured by the imaging device 106 and the operating state of the blast furnace 100 to the blast furnace monitoring device 110 (S108).

(Blast furnace monitoring device according to embodiment)
FIG. 6 is a diagram showing the blast furnace monitoring device 110.

The blast furnace monitoring device 110 includes a monitoring communication section 111, a monitoring storage section 112, a monitoring input section 113, a monitoring output section 114, a monitoring processing section 120, and a learning device 30 that receives input from the image judgment learning device. . The monitoring communication section 111, the monitoring storage section 112, the monitoring input section 113, the monitoring output section 114, and the monitoring processing section 120 are connected to each other via a bus 210. The blast furnace monitoring device 110 has a learning device 30 learned by the image judgment learning device 1, and outputs the operating state of the blast furnace 100 according to the image transmitted from the imaging device 106.

The monitoring communication unit 111 has a wired communication interface circuit such as Ethernet (registered trademark). The monitoring communication unit 111 communicates with the image judgment learning device 1, the imaging device 106, and a higher-level control device (not shown) via the LAN 109.

The monitoring storage unit 112 includes, for example, at least one of a semiconductor storage device, a magnetic tape device, a magnetic disk device, or an optical disk device. The monitoring storage unit 112 stores operating system programs, driver programs, application programs, data, etc. used in processing by the monitoring processing unit 120. For example, the monitoring storage unit 112 may cause the monitoring processing unit 120 to execute an operating state estimation process that estimates the operating state of the blast furnace 100 using the learning device 30 based on the current image transmitted from the imaging device 106. Stores operating state estimation programs, etc. The operating state estimation program may be installed in the monitoring storage unit 112 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like. Further, the monitoring storage unit 112 stores various data used in the operation state estimation process. For example, the monitoring storage unit 112 stores the learning device 30 learned by the image judgment learning device 1.

The monitoring input unit 113 may be any device that can input data, such as a touch panel or a keyboard. The operator can input letters, numbers, symbols, etc. using the monitoring input section 113. When the monitoring input unit 113 is operated by an operator, it generates a signal corresponding to the operation. The generated signal is then supplied to the monitoring processing unit 120 as an operator's instruction.

The monitoring output unit 114 may be any device that can display videos, images, etc., such as a liquid crystal display or an organic EL display. The monitoring output unit 114 displays a video according to the video data supplied from the monitoring processing unit 120, an image according to the image data, and the like. Further, the monitoring output unit 114 may be an output device that prints video, images, characters, etc. on a display medium such as paper.

The monitoring processing unit 120 includes one or more processors and their peripheral circuits. The monitoring processing unit 120 centrally controls the overall operation of the blast furnace monitoring device 110, and is, for example, a CPU. The monitoring processing unit 120 executes processing based on programs (driver programs, operating system programs, application programs, etc.) stored in the monitoring storage unit 112. Further, the monitoring processing unit 120 can execute multiple programs (application programs, etc.) in parallel.

The monitoring processing section 120 includes a current image data acquisition section 121 , a current image input section 122 , an operating state acquisition section 123 , and an operating state output section 124 . Each of these units is a functional module realized by a program executed by a processor included in the monitoring processing unit 120. Alternatively, each of these parts may be implemented in the blast furnace monitoring device 110 as firmware.

(Operation status estimation process by blast furnace monitoring device 110)
FIG. 7 is a flowchart of the operation state estimation process executed by the blast furnace monitoring device 110. The operation state estimation process shown in FIG. 7 is mainly executed by the monitoring processing unit 120 in cooperation with each element of the blast furnace monitoring device 110 based on a program stored in the monitoring storage unit 112 in advance. In addition, since the conditions of the raceway 107 that are visually recognized from about 40 tuyeres 102 are different from each other, the operation state estimation process uses image data transmitted from the imaging device 106 arranged at each of the tuyeres 102. and is executed for each of the tuyeres 102.

First, the current image data acquisition unit 121 acquires current image data indicating the current image captured by the imaging device 106 corresponding to the target tuyere 102 (S111). Next, the current image input unit 122 inputs the current image data acquired by the current image data acquisition unit 121 to the learning device 30 (S112). Next, the operating state acquisition unit 123 obtains the operating state of the blast furnace 100 according to the input of the current image data from the learning device 30 (S113). Then, the operating state output unit 124 outputs an operating state signal indicating the operating state of the blast furnace 100 output from the learning device 30 (S114).

(Operations and effects of the image judgment learning device according to the first embodiment)
Since the image judgment learning device 1 causes the learning device 30 to learn using a group of images showing the falling raw ore state and damage to blast furnace equipment, the blast furnace monitoring device 110 equipped with the learning device 30 can perform judgment without relying on the operator's visual observation. It is possible to monitor the condition of raw ore falling and the presence or absence of damage to blast furnace equipment.

In addition, by using the learning device 30 generated by the image judgment learning device 1, the process of determining whether or not the raw ore has fallen and the presence or absence of damage to the blast furnace equipment does not depend on the operator's subjectivity, so it is consistent and objective. is maintained. The process of determining whether or not the raw ore has fallen off and whether there is damage to the blast furnace equipment is monitored using the learning device 30 generated for each of the approximately 40 tuyeres 102 of the blast furnace 100, so that the processing for determining whether or not the raw ore has fallen off is instantaneously performed for each of the tuyeres 102. Moreover, continuous and highly accurate monitoring becomes possible.

Furthermore, since the learning device 30 performs learning using a group of images, the blast furnace monitoring device 110 equipped with the learning device 30 does not need to use a threshold value for judgment determined by the operator. Even higher consistency is maintained without depending on subjectivity such as.

Furthermore, the blast furnace monitoring device 110 equipped with the learning device 30 can estimate the operating state of the blast furnace with high probability based on the current image captured by the imaging device 106 without requiring visual monitoring by the operator. I can do it.

FIG. 8 is a diagram showing an example of an estimation result by the blast furnace monitoring device 110 equipped with the learning device 30 learned by the image judgment learning device 1. In FIG. 8, the horizontal axis indicates the operational status of the blast furnace in each of the tuyeres 1 to 40, and in order from the left, the respective lance purge status, normal status, and A state requiring attention, a raw ore drop state, and a broken lance state are indicated. In FIG. 8, the left end shows the lance purge state of the first tuyere, and the right end shows the lance damaged state of the 40th tuyere. Furthermore, in FIG. 8, the vertical axis indicates the correct answer rate for each operating state of the blast furnace monitoring device 110 for each of the first to 40th tuyeres.

In the example shown in FIG. 8, the learning device 30 was trained using images captured every 10 ms over a period of one month for each of the first to 40th tuyeres. The correct answer rate for each operating state of the blast furnace monitoring device 110 was determined by the operator based on images captured in a period different from the period in which the learning device 30 learned.

The blast furnace monitoring device 110 equipped with the learning device 30 trained by the image judgment learning device 1 estimates the operating state of the blast furnace 100 with a high correct answer rate of 80% or more regardless of the operating state of the blast furnace 100 at any tuyere. are doing.

(Image judgment learning device according to second embodiment)
FIG. 9 is a diagram showing an image judgment learning device according to the second embodiment. The image judgment learning device 2 according to the second embodiment is arranged in place of the image judgment learning device 1 according to the first embodiment in the blast furnace monitoring system shown in FIG. 30 is output to the blast furnace monitoring device 110.

The image judgment learning device 2 differs from the image judgment learning device 1 in that it includes a processing section 40 instead of the processing section 20. The processing unit 40 differs from the processing unit 20 in that it has an image data acquisition unit 41, a clustering unit 42, a labeling unit 43, and a learning data determination unit 44 instead of the image data acquisition unit 21 to the learning data determination unit 24. . The configurations and functions of the components of the image judgment learning device 2 other than the image data acquisition unit 41 to the learning data determination unit 44 are the same as the configurations and functions of the components of the image judgment learning device 1 with the same reference numerals. A detailed explanation will be omitted here.

(Image judgment learning process by image judgment learning device 2)
FIG. 10 is a flowchart of the image judgment learning process executed by the image judgment learning device 2. The image judgment learning process shown in FIG. 10 is mainly executed by the processing unit 40 in cooperation with each element of the image judgment learning device 2 based on a program stored in the storage unit 12 in advance. In addition, since the states of the raceway 107 that are visually recognized from about 40 tuyeres 102 are different from each other, the image judgment learning process uses image data transmitted from the imaging device 106 arranged at each of the tuyeres 102. and is executed for each of the tuyeres 102.

First, the image data acquisition unit 41 acquires run purge time information indicating the run purge time during which pulverized coal is not supplied from the tuyere 102 into the blast furnace 100 (S201). If the run purge time information is stored in the storage unit 12 in advance, the image data acquisition unit 41 acquires the run purge time information from the storage unit 12. If the run purge time information is not stored in the storage unit 12 in advance, the image data acquisition unit 41 acquires the run purge time information from a higher-level control device (not shown) via the LAN 109.

Next, the image data acquisition unit 41 extracts a plurality of image data representing a plurality of images captured at different times by the imaging device 106 corresponding to the target tuyere 102, excluding images captured during the run purge time. Acquire (S202). The image data acquisition unit 41 identifies the run purge time by referring to the run purge time information, and acquires a plurality of image data by excluding images captured during the run purge time.

Next, the clustering unit 42 clusters the images corresponding to the image data acquired by the image data acquisition unit 41 into four image groups (S203). Similar to the clustering unit 22, the clustering unit 42 clusters images corresponding to the image data acquired by the image data acquisition unit 41 into four image groups using a known clustering method such as the k-means method.

Next, the labeling unit 43 associates each of the four image groups clustered by the clustering unit 42 with the operating state of the blast furnace 100 (S204). Specifically, the labeling unit 23 uses representative images associated with each operating state of the blast furnace 100 to label four images based on the similarity (distance) between each of the four image groups and the representative image. Each of the groups is associated with an operating state of the blast furnace 100. There are four operating states of the blast furnace 100 associated with the labeling unit 43: a normal state, a caution-required state, a dropped raw ore state, and a lance broken state, excluding the lance purge state. Further, like the labeling unit 23, the labeling unit 43 may set a certain image group to a “not applicable” state.

Next, the learning data determining unit 44 determines all of the operational states of the blast furnace 100 associated with the four image groups clustered in the process of S203 as a normal state, a caution-required state, a dropped raw ore state, and a broken lance state. It is determined whether it is included (S205).

When it is determined that the operating state associated with the four image groups does not include any of the four states such as a normal state (S205-NO), the learning data determining unit 44 increases the clustering number N by one. and 5 (S206).

Thereafter, the processes of S203 to S206 are repeated while increasing the clustering number N by one until it is determined that the operating state associated with the N image groups includes all four states (S205-YES). .

When the learning data determination unit 44 determines that the operating state associated with the N image groups includes all four states (S205-YES), the learning data determining unit 44 selects the plurality of images clustered into the image group in the process of S203. The image data corresponding to is determined as learning data (S207).

The processing in S208 to S209 is similar to the processing in S107 to S108, so a detailed explanation will be omitted here.

(Operations and effects of the image judgment learning device according to the second embodiment)
Since the image judgment learning device 2 trains the learning device 30 using image data excluding images captured during the run purge time, the learning time of the learning device 30 can be shorter than that of the image judgment learning device 1. . Further, since the image judgment learning device 2 causes the learning device 30 to learn using image data excluding images captured during the run purge time, it is possible to improve the accuracy of clustering and, by extension, the accuracy of learning. For example, even if an image is acquired in which it is difficult to distinguish between the run purge state and other operating states, if the images in the run purge state are excluded, reliable learning about the other operating states can be performed.

(Image judgment learning device according to third embodiment)
FIG. 11 is a diagram showing an image judgment learning device according to the third embodiment. The image judgment learning device 3 according to the third embodiment is arranged in place of the image judgment learning device 1 according to the first embodiment in the blast furnace monitoring system shown in FIG. 30 is output to the blast furnace monitoring device 110.

The image judgment learning device 3 differs from the image judgment learning device 2 in that it includes a processing section 50 instead of the processing section 40. The processing section 50 is different from the processing section 40 in that it has a clustering section 52, a labeling section 53, and a learning data determining section 54 instead of the clustering section 42 to the learning data determining section 44. The configurations and functions of the components of the image judgment learning device 3 other than the clustering unit 52 to the learning data determining unit 54 are the same as those of the image judgment learning device 2 with the same reference numerals, so they will not be described here. Detailed explanation will be omitted.

(Image judgment learning process by image judgment learning device 3)
FIG. 12 is a flowchart of the image judgment learning process executed by the image judgment learning device 3. The image judgment learning process shown in FIG. 12 is mainly executed by the processing unit 50 in cooperation with each element of the image judgment learning device 3 based on a program stored in the storage unit 12 in advance. In addition, since the states of the raceway 107 that are visually recognized from about 40 tuyeres 102 are different from each other, the image judgment learning process uses image data transmitted from the imaging device 106 arranged at each of the tuyeres 102. and is executed for each of the tuyeres 102.

The processing in S301 to S302 is similar to the processing in S201 to S202, so a detailed explanation will be omitted here.

Next, the clustering unit 52 clusters the images corresponding to the image data acquired by the image data acquisition unit 41 into three image groups (S303). Similar to the clustering unit 42, the clustering unit 52 clusters images corresponding to the image data acquired by the image data acquisition unit 41 into three image groups using a known clustering method such as the k-means method.

Next, the labeling unit 53 associates each of the three image groups clustered by the clustering unit 52 with the operating state of the blast furnace 100 (S304). Specifically, the labeling unit 23 uses representative images associated with each operating state of the blast furnace 100 to label three images based on the degree of similarity (distance) between each of the three image groups and the representative image. Each of the groups is associated with an operating state of the blast furnace 100. The operating states of the blast furnace 100 associated with the labeling unit 43 are the normal state, the raw ore drop state, and the lance broken state, and the lance purge state and the caution-required state are excluded.

Next, when there is a group of images that are not associated with any of the operating states of the blast furnace 100 in the process of S304, the labeling unit 53 requests the image group that is not associated with any one of the operating states of the blast furnace 100. It is associated with the attention state (S305). Specifically, the labeling unit 53 determines whether there is a group of images that are not associated with any one of the operating states of the blast furnace 100 in the process of S304. When it is determined in the process of S304 that there is a group of images that are not associated with any of the operating states of the blast furnace 100, the labeling unit 53 labels the image group that is not associated with any one of the operating states of the blast furnace 100. It is associated with the caution-required state (S305).

Next, the learning data determining unit 54 determines whether the operational status of the blast furnace 100 associated with the three image groups clustered in the process of S304 includes all of the normal status, raw ore drop status, and lance broken status. is determined (S306).

When it is determined that the operating states associated with the three image groups do not include any of the three states such as the normal state (S306-NO), the learning data determining unit 54 increases the clustering number N by one. and 4 (S307).

Thereafter, the processes of S303 to S307 are repeated while increasing the clustering number N by one until it is determined that the operating state associated with the N image groups includes all three states (S306-YES). .

When the learning data determining unit 54 determines that the operating state associated with the N image groups includes all three states (S306-YES), the learning data determination unit 54 selects the plurality of images clustered into the image group in the process of S303. The image data corresponding to is determined as learning data (S308).

The processing in S309 to S310 is similar to the processing in S208 to S209, so a detailed explanation will be omitted here.

(Operations and effects of the image judgment learning device according to the third embodiment)
The image judgment learning device 3 associates a group of images that have not been associated with any of the operational states of the blast furnace 100 with the caution-required state, thereby labeling the caution-required state in which it is difficult for even a skilled operator to select a representative image. can be excluded from processing. The image judgment learning device 3 can exclude from the labeling process caution-required states in which it is difficult for even a skilled operator to select a representative image, so that consistency and objectivity can be improved from the image judgment learning devices 1 and 2. can be further improved.

(Modified example of image judgment learning device according to embodiment)
Image judgment learning devices 1 to 3 use representative images associated with each operating state of the blast furnace 100 to determine at least three image groups based on the similarity (distance) between each of the at least three image groups and the representative image. The image group is labeled by associating each image group with any one of the operating states of the blast furnace 100. However, the image judgment learning device according to the embodiment may label each of the at least three image groups using other methods. For example, the image judgment learning device according to the embodiment may label the image group based on a record of the operating state of the blast furnace 100 stored in a higher-level control device (not shown) or the like. The image judgment learning device according to the embodiment determines the most frequent operating state among the operating states of the blast furnace 100 at each time when images included in each image group are acquired, based on the record of the operating state of the blast furnace 100. The operating state of the blast furnace 100 may be determined.

Furthermore, the image judgment learning devices 1 to 3 may cause the learning device 30 to learn multiple times. For example, the image judgment learning devices 1 to 3 may cause the learning device 30 to learn at every predetermined learning cycle, and output the learned learning device 30 to the blast furnace monitoring device 110. The learning cycle can be, for example, one year.

Further, the image judgment learning devices 1 to 3 increase the number of clusterings when the operating state of the blast furnace 100 associated with the clustered image group does not include the desired operating state. However, the image judgment learning device according to the embodiment may increase the number of clusterings after performing clustering a predetermined number of times. By performing clustering multiple times, randomly allocated image clusters can be clustered in different states, making it possible to cluster images to include desired operating states without increasing the number of clusters. There is.

Further, the image judgment learning devices 1 to 3 use the first abnormal state as a fallen raw ore state and the second abnormal state as a broken lance state, but the image judgment learning device according to the embodiment damage to blast furnace equipment may be used as an abnormal condition.

1 to 3 Image judgment learning device 21, 41 Image data acquisition section 22, 42, 52 Clustering section 23, 43, 53 Labeling section 24, 44, 54 Learning data determining section 25 Learning processing section 26 Learning device output section 30 Learning device 100 Blast Furnace 101 Furnace Body 102 Tuyere 103 Hot Air Supply Pipe 104 Pulverized Coal Supply Pipe 105 Observation Window 106 Imaging Device 107 Raceway 108 Imaging Unit 109 LAN
110 Blast furnace monitoring device

Claims (7)

Acquire a plurality of image data showing the internal state of the blast furnace taken at different times by an imaging device having an imaging unit placed opposite to one of the tuyeres that supplies pulverized coal along with gas to the inside of the blast furnace. an image data acquisition unit,
a clustering unit that clusters images corresponding to each of the plurality of image data into at least three image groups;
a labeling unit that associates each of the at least three image groups with an operating state of the blast furnace;
When it is determined that the operational state of the blast furnace associated with the at least three image groups includes a normal state, a first abnormal state, and a second abnormal state, the normal state, the first abnormal state, and the second abnormal state. a learning data determining unit that determines, as learning data , a plurality of image data indicating the internal state of the blast furnace clustered into a group of images associated with any one of the images;
a learning processing unit that causes a learning device to learn the relationship between the internal state of the blast furnace and the operational state of the blast furnace using the learning data;
a learning device output unit that outputs a learning device that has learned the relationship between the internal state of the blast furnace and the operating state of the blast furnace;
The first abnormal state is a state in which a phenomenon in which ore that is not completely melted falls,
The second abnormal state is a state in which the pulverized coal supply pipe that supplies pulverized coal to the tuyere is damaged.
An image judgment learning device characterized by: When determining that the operational state of the blast furnace associated with the at least three image groups does not include at least one of a normal state, a first abnormal state, and a second abnormal state, the clustering unit selects images to be clustered. The image judgment learning device according to claim 1, wherein the number of groups is increased and the clustering process is executed again. The image data acquisition unit acquires run purge time information indicating a run purge time during which pulverized coal is not supplied into the blast furnace from the tuyere, and extracts images captured during the run purge time to obtain the plurality of image data. The image judgment learning device according to claim 1 or 2 , wherein the image judgment learning device obtains. When there is a group of images that are not associated with any of the normal state, the first abnormal state, and the second abnormal state, the labeling unit may label the group of images in a manner that indicates that there is a risk that an abnormality may occur in the operation of the blast furnace. The image judgment learning device according to any one of claims 1 to 3 , wherein the image judgment learning device is associated with a caution-required state indicating a condition requiring attention. A learning device that outputs an operating state of the blast furnace according to an input image learned by the image judgment learning device according to any one of claims 1 to 4 ;
a current image data acquisition unit that acquires current image data indicating a current image captured by the imaging device;
a current image input unit that inputs the current image data to the learning device;
an operating state acquisition unit that obtains an operating state of the blast furnace according to input of the current image data from the learning device;
an operating state output unit that outputs an operating state signal indicating the operating state of the blast furnace output from the learning device;
A blast furnace monitoring device characterized by having: Acquire a plurality of image data showing the internal state of the blast furnace taken at different times by an imaging device having an imaging unit placed opposite to one of the tuyeres that supplies pulverized coal along with gas to the inside of the blast furnace. death,
clustering images corresponding to each of the plurality of image data into at least three image groups;
associating each of the at least three image groups with an operating state of the blast furnace;
When it is determined that the operational state of the blast furnace associated with the at least three image groups includes a normal state, a first abnormal state, and a second abnormal state, the normal state, the first abnormal state, and the second abnormal state. determining a plurality of image data showing the internal state of the blast furnace clustered into a group of images associated with any one of the images as learning data;
causing a learning device to learn the relationship between the internal state of the blast furnace and the operational state of the blast furnace using the learning data;
outputting a learning device that has learned the relationship between the internal state of the blast furnace and the operational state of the blast furnace,
The first abnormal state is a state in which a phenomenon in which ore that is not completely melted falls,
The second abnormal state is a state in which the pulverized coal supply pipe that supplies pulverized coal to the tuyere is damaged.
An image judgment learning method characterized by the following. Acquire a plurality of image data showing the internal state of the blast furnace taken at different times by an imaging device having an imaging unit placed opposite to one of the tuyeres that supplies pulverized coal along with gas to the inside of the blast furnace. death,
clustering images corresponding to each of the plurality of image data into at least three image groups;
associating each of the at least three image groups with an operating state of the blast furnace;
When it is determined that the operational state of the blast furnace associated with the at least three image groups includes a normal state, a first abnormal state, and a second abnormal state, the normal state, the first abnormal state, and the second abnormal state. determining a plurality of image data showing the internal state of the blast furnace clustered into a group of images associated with any one of the images as learning data;
causing a learning device to learn the relationship between the internal state of the blast furnace and the operational state of the blast furnace using the learning data;
An image judgment learning program that causes a computer to perform processing that outputs a learning device that has learned the relationship between the internal state of the blast furnace and the operating state of the blast furnace,
The first abnormal state is a state in which a phenomenon in which ore that is not completely melted falls,
The second abnormal state is a state in which the pulverized coal supply pipe that supplies pulverized coal to the tuyere is damaged.
An image judgment learning program characterized by:

Images