WO2024106008A1

WO2024106008A1 - Collapse detection system and collapse detection method

Info

Publication number: WO2024106008A1
Application number: PCT/JP2023/034285
Authority: WO
Inventors: 信治岩下; 慶子青山; 駿郡司
Original assignee: 三菱重工業株式会社
Priority date: 2022-11-17
Filing date: 2023-09-21
Publication date: 2024-05-23

Abstract

The collapse detection system disclosed herein comprises: an acquisition unit that acquires, at a first prescribed cycle, captured images of an object to be incinerated that is deposited in a feeder of an incineration facility and pushed out toward a combustion chamber; a first calculation unit that calculates a representative value of brightness based on a first image acquired by the acquisition unit; a second calculation unit that calculates a representative value of brightness based on two or more images captured within a period of time for one collapse of the object to be incinerated, among a plurality of images in time series that were acquired by the acquisition unit before the first image; and a determination unit that performs determination related to the collapse on the basis of the representative value of brightness calculated by the first calculation unit and the representative value of brightness calculated by the second calculation unit.

Description

Collapse detection system and collapse detection method

This disclosure relates to a collapse detection system and a collapse detection method. This application claims priority to Japanese Patent Application No. 2022-183989, filed on November 17, 2022, the contents of which are incorporated herein by reference.

Patent Document 1 discloses a supply amount detection system including an imaging device configured to capture an image of solid fuel before it accumulates in a feeder section of an incinerator and falls into a combustion chamber, and a detection device that detects the amount of the solid fuel supplied to the combustion chamber based on the image captured by the imaging device. In this supply amount detection system, the amount of the solid fuel supplied to the combustion chamber is detected based on the difference between a first luminance, which is the luminance of the image at a first timing, and a second luminance, which is the luminance of the image at a second timing that is later than the first timing and is lower than the first luminance. Specifically, the detection device divides the image into a plurality of partition images, counts the number of partition images in which the difference between the first luminance and the second luminance in each of the plurality of partition images exceeds a preset threshold, and detects that the amount of the solid fuel supplied to the combustion chamber is excessive when the counted number exceeds a preset number.

Patent Document 2 discloses a fluidized bed incineration apparatus having a dust feeder and a chute connecting the dust feeder's dust outlet and the dust supply port of a fluidized bed incinerator, characterized in that a television camera is attached at a position where the falling of the dust from the dust feeder's dust outlet can be observed, and a calculation means is provided for calculating the amount of falling trash and the amount of heat generated by the trash based on the image from the attached television camera. In this fluidized bed incineration apparatus, the amount of trash is detected by a processing flow including: (a) taking in an image; (b) binarizing the image; (c) recognizing the contour; (d) calculating the area within the contour; (e) calculating the center of gravity within that area; (f) storing the center of gravity and area; and (g) multiplying the area by the distance traveled by the center of gravity calculated last time and the center of gravity calculated this time.

Patent Document 3 discloses a combustion field observation device that can visualize and observe a combustion field such as the inside of a boiler furnace during operation. This observation device has a means for converting an image obtained by photographing the combustion field to grayscale, a means for adjusting the contrast of an image obtained by photographing the combustion field, or a means for converting an image obtained by photographing the combustion field to grayscale and then adjusting the contrast, and performs a specified process on an image photographed at a certain timing to generate information for visualizing the combustion field.

Patent No. 6979482 Japanese Patent Application Laid-Open No. 9-060842 JP 2019-196845 A

However, because complex events occur within the combustion chamber, it can be difficult to properly detect the collapse of incinerated materials. However, there is hope for improving the accuracy of detecting the collapse of incinerated materials in order to more appropriately control the combustion chamber.

The present disclosure has been made to solve the above problems, and aims to provide a collapse detection system and collapse detection method that can improve the accuracy of detecting the collapse of materials to be incinerated.

In order to solve the above problem, the collapse detection system disclosed herein comprises an acquisition unit that acquires images of materials to be incinerated that are piled up in a feeder of an incineration facility and pushed toward a combustion chamber at a first predetermined period; a first calculation unit that calculates a representative value of brightness based on the first images acquired by the acquisition unit; a second calculation unit that calculates a representative value of brightness based on two or more images captured within the time it takes for one collapse of the materials to be incinerated, among a plurality of images in a time series acquired by the acquisition unit before the first image; and a determination unit that makes a determination regarding the collapse based on the representative value of brightness calculated by the first calculation unit and the representative value of brightness calculated by the second calculation unit.

The collapse detection method disclosed herein includes one or more computers acquiring images of materials to be incinerated that have accumulated in a feeder of an incineration facility and are being pushed toward a combustion chamber at a first predetermined period, calculating a representative value of brightness based on the acquired first image, calculating a representative value of brightness based on two or more images acquired within the time it takes for one collapse of the materials to occur among a plurality of images in a time series acquired before the first image, and making a determination regarding the collapse based on the representative value of brightness based on the first image and the representative value of brightness based on the two or more images.

The collapse detection system according to the present disclosure includes an acquisition unit that acquires images of materials to be incinerated that have accumulated in a feeder of an incineration facility and are being pushed toward a combustion chamber at a first predetermined period, and a collapse detection unit that makes a determination regarding the collapse based on a first input element that is a first image acquired by the acquisition unit, and a second input element that is two or more images captured within the time it takes for one collapse of the materials to occur among a plurality of images in a time series acquired by the acquisition unit prior to the first image.

The present disclosure provides a collapse detection system and a collapse detection method that can improve the accuracy of detecting the collapse of materials to be incinerated.

1 is a schematic diagram showing an overall configuration of an incineration facility according to an embodiment of the present disclosure. FIG. 1 is a functional block diagram of an information processing device according to an embodiment of the present disclosure. 4 is a diagram showing an example of an image to be determined by a first determination unit according to the first embodiment of the present disclosure; FIG. 13 is a diagram showing an example of an image to be determined by a third determination unit according to the first embodiment of the present disclosure; FIG. 6A to 6C are diagrams illustrating an example of a time-series determination result by a first determination unit according to the first embodiment of the present disclosure. A figure showing an example of a list of images used for learning by the second judgment trained model according to the first embodiment of the present disclosure. 6A to 6C are diagrams illustrating an example of a time-series determination result by a second determination unit according to the first embodiment of the present disclosure. 4 is a diagram showing an example of an image to be determined by a disturbance determination unit according to the first embodiment of the present disclosure; FIG. 1A to 1C are diagrams illustrating an example of a low deviation image and a high deviation image used for learning by a trained model for disturbance determination according to the first embodiment of the present disclosure. A figure showing a list of images used for learning by the trained model for disturbance determination according to the first embodiment of the present disclosure, in a format corresponding to the magnitude of the deviation in brightness. 5 is a diagram showing an example of a time-series determination result by a disturbance determination unit according to the first embodiment of the present disclosure; FIG. 13A to 13C are diagrams illustrating an example of a time-series determination result by a third determination unit according to the first embodiment of the present disclosure. 5 is a flowchart illustrating an example of an operation of the information processing device according to the first embodiment of the present disclosure. 11A to 11C are diagrams illustrating an example of a time-series determination result by a first collapse determination unit and a second collapse determination unit according to the first embodiment of the present disclosure. FIG. 11 is a diagram for explaining binarized data according to a second embodiment of the present disclosure. 13 is a diagram for conceptually explaining a method for calculating a similarity by a second collapse determiner according to a second embodiment of the present disclosure. FIG. 10 is a flowchart illustrating an example of an operation of the information processing device according to the second embodiment of the present disclosure. 13A to 13C are diagrams illustrating an example of a time series of determination results by a first collapse determination unit and a second collapse determination unit according to a second embodiment of the present disclosure. FIG. 13 is a functional block diagram of an information processing device according to a third embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of an image that is a target of calculation by a first calculator according to a third embodiment of the present disclosure. A figure showing an example of a list of images used for learning by the fourth judgment trained model according to the third embodiment of the present disclosure. 13 is a table showing a numerical range of the sum of the luminance change amounts that is used as a criterion for judgment by a third collapse judgment unit according to a third embodiment of the present disclosure, and judgment results of a fourth and fifth judgment unit. 13 is a flowchart illustrating an example of an operation of the information processing device according to the third embodiment of the present disclosure. FIG. 2 is a hardware configuration diagram illustrating a configuration of a computer according to an embodiment of the present disclosure.

Below, we will explain the implementation of the incineration equipment and the collapse detection system with reference to the attached drawings.

<First embodiment of incineration facility>
The incineration facility 100 is a stoker-type waste incinerator in which, for example, urban waste, industrial waste, biomass, or the like is incinerated. Hereinafter, the material to be incinerated may be referred to as "waste." That is, in this embodiment, the waste is fuel for causing a combustion reaction in the incineration facility. As shown in FIG. 1, the incineration facility 100 includes, for example, a hopper 102, a feeder 104, a furnace body 108, an extrusion device 110, an air supply device 112, a heat recovery boiler 114, a cooling tower 116, a dust collector 118, a chimney 120, and a collapse detection system 1.

The feeder 104 is a passageway that extends toward the combustion chamber R of the furnace body 108. Waste Fg fed from the hopper 102 is introduced into the feeder 104 and temporarily piles up. The furnace body 108 has an internal combustion chamber R for incinerating the waste Fg. If the direction in which the waste Fg is transported within the furnace body 108 is defined as the transport direction W1 (the left-right direction in FIG. 1), then the downstream end 121 of the feeder 104 downstream of the transport direction W1 is connected to the receiving port 122 of the combustion chamber R.

The push-out device 110 has a push-out arm 124 for pushing out the waste Fg accumulated in the feeder 104 into the combustion chamber R through the receiving port 122. The push-out arm 124 is movable (advanceably and retreatably) within the feeder 104 from the upstream side to the downstream side and from the downstream side to the upstream side in the conveying direction W1. In this embodiment, the push-out arm 124 reciprocates within the feeder 104 in the conveying direction W1, and intermittently supplies the waste Fg into the combustion chamber R. In this embodiment, the push-out arm 124 is an example of a device S to be controlled.

The furnace body 108 includes a grate 126 (stoker) into which the waste Fg pushed into the combustion chamber R through the receiving port 122 falls. The grate 126 corresponds to the floor of the combustion chamber R. The grate 126 moves the waste Fg on the grate 126 in a direction away from the receiving port 122 (downstream in the conveying direction W1). The grate 126 is an example of a controlled device S. The combustion chamber R also includes a drying area 128, a combustion area 130, and a post-combustion area 132, which are arranged in order from the upstream side to the downstream side in the conveying direction W1. The drying area 128 dries the waste Fg by the heat in the combustion chamber R. The combustion area 130 burns the waste Fg by raising a flame 131. The post-combustion area 132 completely burns the remaining burnt waste that was not completely burned in the combustion area 130. The waste Fg that has been dried, burned, and post-combusted in the combustion chamber R becomes ash 135, which falls through an ash chute 146 downstream of the post-combustion area 132 and is discharged outside the furnace body 108.

The air supply device 112 supplies the combustion chamber R with primary air used for burning the waste Fg and secondary air used to reduce the concentration of unburned gases such as carbon monoxide generated by the burning of the waste Fg. The air supply device 112 has an air supply pipe 136, a blower 138 provided in the air supply pipe 136, and a first flow control valve 140 and a second flow control valve 142 provided in the air supply pipe 136. A portion of the air pressurized from the blower 138 and flowing through the air supply pipe 136 is supplied into the combustion chamber R from the lower part of the combustion chamber R through the grate 126 while its flow rate is adjusted by the first flow control valve 140 as primary air. The remaining portion of the air flowing through the air supply pipe 136 is supplied to the upper part of the combustion chamber R from the side wall of the combustion chamber R while its flow rate is adjusted by the second flow control valve 142 as secondary air. In this embodiment, for example, primary air is supplied to each of the drying area 128, the combustion area 130, and the post-combustion area 132 of the combustion chamber R, and secondary air is supplied to the upper side of the combustion area 130. In this embodiment, the blower 138, the first flow control valve 140, and the second flow control valve 142 are all examples of the controlled device S.

The heat recovery boiler 114, the temperature reducing tower 116, the dust collector 118, and the chimney 120 are each provided in a flue 144 through which exhaust gas 143 generated by burning waste Fg in the combustion chamber R flows. The exhaust gas 143 flows through the heat recovery boiler 114, the temperature reducing tower 116, the dust collector 118, and the chimney 120 in that order. The heat recovery boiler 114 generates steam from the thermal energy of the exhaust gas 143. The temperature reducing tower 116 lowers the temperature of the exhaust gas 143 that has passed through the heat recovery boiler 114. The dust collector 118 collects fly ash contained in the exhaust gas 143 that has passed through the temperature reducing tower 116. The chimney 120 exhausts the exhaust gas 143 that has passed through the dust collector 118 to the outside of the incineration equipment 100. The steam generated in the heat recovery boiler 114 is supplied to, for example, a steam turbine (not shown) located outside the incineration facility 100.

<Collapse detection system>
The collapse detection system 1 detects that the garbage Fg accumulated in the feeder 104 has been pushed by the push-out arm 124 toward the combustion chamber R and supplied to the combustion chamber R. Specifically, the collapse detection system 1 detects the collapse of the garbage Fg from inside the feeder 104 into the combustion chamber R. The "collapse" here means, for example, that a certain amount of garbage Fg is supplied at once from the garbage Fg accumulated in the feeder 104 to the combustion chamber R. In this embodiment, collapse is classified into a first-scale collapse and a second-scale collapse that is larger in scale than the first-scale collapse.

The first-scale collapse includes a case where the layer structure of about one-third of the total garbage Fg accumulated in the feeder 104 in the width direction of the furnace body 108 (direction perpendicular to the conveying direction W1) cannot be visually recognized in the infrared image captured by the imaging device 2 described below, and part of the garbage Fg after the collapse scatters inside the combustion chamber R. The first-scale collapse also includes a case where the layer structure of more than one-third but less than two-thirds of the total garbage Fg accumulated in the feeder 104 in the width direction of the furnace body 108 cannot be visually recognized in the infrared image, and part of the garbage Fg after the collapse does not scatter inside the combustion chamber R.

On the other hand, the second scale collapse includes a case where the layer structure of the garbage Fg in more than two-thirds of the width of the furnace body 108 of the total garbage Fg accumulated in the feeder 104 in the infrared image becomes impossible to see, and part of the garbage Fg after the collapse scatters inside the combustion chamber R. In addition, the second scale collapse also includes a case where the garbage Fg falls onto the flame 131 in the visible light image captured by the imaging device 2 described below, causing the flame 131 to disappear in an area of more than one-third of the width of the furnace body 108.

Therefore, in this embodiment, the collapse detection system 1 detects the scale (amount) of the garbage Fg that has collapsed into the combustion chamber R. As shown in Figures 1 and 2, the collapse detection system 1 includes, for example, an imaging device 2 and an information processing device 4.

[Imaging device]
The imaging device 2 captures an image of the inside of the combustion chamber R so that the garbage Fg accumulated in the feeder 104 is reflected. The image of the garbage Fg captured by the imaging device 2 is transmitted to the information processing device 4 in real time. The imaging device 2 is arranged in the furnace body 108 so as to capture an image of the front surface Fr of the garbage Fg facing the combustion chamber R before it collapses into the combustion chamber R. Specifically, the imaging device 2 is provided at the bottom 145 of the furnace body 108 located downstream of the post-combustion region 132 in the combustion chamber R in the conveying direction W1. Note that the imaging device 2 may be provided at a location other than the bottom 145 of the furnace body 108 as long as it is possible to capture an infrared image and a visible light image of the front surface Fr of the garbage Fg.

In this embodiment, the imaging device 2 has an infrared camera 5 capable of capturing infrared images and a visible light camera 6 capable of capturing visible light images (see FIG. 1). The imaging device 2 is capable of capturing infrared images and visible light images of the front surface Fr of the garbage Fg protruding from the receiving port 122 of the combustion chamber R toward the downstream side in the conveying direction W1. The infrared camera 5 captures the front surface Fr of the garbage Fg in a wavelength band of, for example, 3.8 μm to 4.2 μm, and generates an infrared image. Since the infrared camera 5 captures images in the above wavelength band, it can transmit the flame 131, and the flame 131 is prevented from being reflected in the generated infrared image. The visible light camera 6 captures the front surface Fr of the garbage Fg in a predetermined wavelength band in the visible wavelength range, and generates a visible light image. Since the visible light camera 6 captures images in the above wavelength band, it cannot transmit the flame 131, and the flame 131 is mainly reflected in the generated visible light image.

[Information processing device]
The information processing device 4 detects information about the garbage Fg supplied from the feeder 104 to the combustion chamber R based on the image captured by the imaging device 2. As shown in FIG. 2, the information processing device 4 includes, for example, an acquisition unit 40, a collapse detection unit 41, a control unit 80, and a storage unit 90.

(Configuration of Acquisition Unit)
The acquisition unit 40 acquires images over time by receiving images transmitted from the imaging device 2 in real time. In this embodiment, the acquisition unit 40 acquires infrared images from the imaging device 2 at a first predetermined cycle, and acquires visible light images from the imaging device 2 at a second predetermined cycle. The first predetermined cycle and the second predetermined cycle are determined based on, for example, the frame rate (fps: flames per second) of the imaging device 2. The acquisition unit 40 sends the acquired infrared images and visible light images to the collapse detection unit 41.

(Configuration of collapse detection unit)
The collapse detection unit 41 detects, based on the infrared image and the visible light image received from the acquisition unit 40, that the waste Fg has collapsed from the feeder 104 into the combustion chamber R, and the scale (amount) of the waste Fg that has collapsed and been supplied to the combustion chamber R. The collapse detection unit 41 has, for example, a first calculation unit 50, a second calculation unit 55, a third calculation unit 60, a fourth calculation unit 65, and a determination unit 70.

(First Calculation Unit)
The first calculation unit 50 receives an infrared image from the images received from the acquisition unit 40, and calculates a first feature amount based on the received infrared image. In this embodiment, the first feature amount is a representative value of luminance based on one infrared image (e.g., the most recent infrared image) received by the first calculation unit 50. Hereinafter, the infrared image used by the first calculation unit 50 to calculate the representative value of luminance is referred to as the "first infrared image". That is, the first calculation unit 50 calculates the representative value of luminance based on the first infrared image acquired by the acquisition unit 40. The first infrared image is an example of the first image. In this embodiment, the first calculation unit 50 calculates the representative value of luminance in a specific target area that is a part of the first infrared image. Hereinafter, the target area is referred to as the "first target area 42".

As shown in FIG. 3, the first calculation unit 50 calculates a representative value of brightness for the first target area 42, which is an area in the first infrared image where the dust Fg accumulated in the feeder 104 is mainly reflected. In this embodiment, the first target area 42 that is the target of calculation by the first calculation unit 50 is divided into multiple areas. FIG. 3 shows an example in which the first target area 42 is equally divided into nine areas in a 3×3 matrix. Note that the first target area 42 is not limited to being equally divided into nine areas in a matrix, and may be divided into 2 to 8 areas or 10 or more areas. Hereinafter, the first target area 42 shown in FIG. 3 will be referred to as the "first area 42a", "second area 42b", "third area 42c", "fourth area 42d", "fifth area 42e", "sixth area 42f", "seventh area 42g", "eighth area 42h", and "ninth area 42i" in order from the upper left to the lower right. For example, the first calculation unit 50 calculates the average value of the luminance of each of the first region 42a to the ninth region 42i in the first target region 42 in the first infrared image, and further averages the calculated nine luminance average values to calculate the average value of the entire first target region 42 as the representative value of the luminance based on the first infrared image. Note that the representative value calculated by the first calculation unit 50 is not limited to the average value, and may be, for example, a statistical value such as a median. In other words, the method of calculating the representative value by the first calculation unit 50 is not necessarily limited to the above. The first calculation unit 50 sends the calculated representative value of the luminance in the first target region 42 to the determination unit 70.

(Second Calculation Unit)
The second calculation unit 55 receives an infrared image from the images received from the acquisition unit 40, and calculates a second feature amount based on the received infrared images. In this embodiment, the second feature amount is a representative value of brightness based on the infrared images received by the second calculation unit 55. Specifically, the second calculation unit 55 calculates a representative value of brightness based on the infrared images captured within the time required for one collapse of the garbage Fg among two or more infrared images in a time series acquired before the first infrared image used by the first calculation unit 50 to calculate the representative value. The "time required for one collapse" here refers to, for example, a time obtained by averaging the acquired multiple times after a time at which a human judges that one collapse of the garbage Fg has occurred is acquired multiple times. Specifically, the time required for one collapse of the garbage Fg is defined by calculating the average number of frames from the distribution of the number of frames (number of images) of the imaging device 2 indicating one collapse.

The two or more infrared images used by the second calculation unit 55 to calculate the representative brightness value are infrared images captured at a predetermined time interval from each other. Hereinafter, the time interval between the two or more infrared images is referred to as the "first time". In this embodiment, the second calculation unit 55 calculates the representative brightness value based on each of the infrared images of the number of frames acquired per unit time by the acquisition unit 40. Therefore, the first time is the above-mentioned first predetermined period. The first time is, for example, less than one second long.

In this embodiment, the two or more infrared images are infrared images captured before the first infrared image for a second time period that is at least twice the first time period. The second calculation unit 55 calculates a representative value of brightness based on the multiple infrared images, for example, three or more (four or more) infrared images as the two or more infrared images. Here, when the time required for one collapse of the garbage Fg is S, the first predetermined period is _Ti , and the number of infrared images (number of frames) used in the calculation is _Ni , the following formula (i) is established.
S/2<T _i ×N _i ... (i)
That is, the multiple infrared images are multiple images acquired over a time period longer than at least half the time period S. In this embodiment, the multiple infrared images are acquired over a time period S. Hereinafter, each of the multiple infrared images used by the second calculator 55 to calculate the representative value of brightness will be referred to as a "second infrared image." The second infrared image is an example of the second image.

The second calculation unit 55 calculates a representative value of the luminance of a specific target region that is a part of the second infrared image. The target region that the second calculation unit 55 calculates is the same region as the first target region 42 described above. The second calculation unit 55 calculates the average value of the luminance of each of the first region 42a to the ninth region 42i of the first target region 42, and calculates the average value of the entire first target region 42 by further averaging the calculated nine average values of luminance. Furthermore, the second calculation unit 55 calculates the average value of the entire first target region 42 as a representative value based on the multiple second infrared images. Note that the representative value calculated by the second calculation unit 55 is not limited to the average value, and may be a statistical value such as a median. In other words, the method of calculating the representative value by the second calculation unit 55 is not necessarily limited to the above. The second calculation unit 55 sends the representative value of the luminance of the entire first target region 42 based on the multiple second infrared images to the determination unit 70.

(Third Calculation Unit)
The third calculation unit 60 receives a visible light image from the images received from the acquisition unit 40, and calculates a third feature amount based on the received visible light image. In this embodiment, the third feature amount is a representative value of luminance based on one visible light image (e.g., the most recent visible light image) received by the third calculation unit 60. Hereinafter, the visible light image used by the third calculation unit 60 to calculate the representative value of luminance is referred to as a "first visible light image". That is, the third calculation unit 60 calculates the representative value of luminance based on the first visible light image acquired by the acquisition unit 40. The first visible light image is another example of the first image. An example of a visible light image received by the third calculation unit 60 is shown in FIG. 4. In FIG. 4, the first visible light image (a) is shown in monotone (black and white display) for convenience of illustration. As shown in FIG. 4, in this embodiment, the third calculation unit 60 generates an image (b) by extracting only the red component from the first visible light image (a) among the monochromatic components, and calculates a representative value of luminance in a specific target region that is a part of the image (b) from which only the red component has been extracted. Hereinafter, the image from which only the red component has been extracted from the first visible light image by the third calculation unit 60 is referred to as a "monochromatic component image." In addition, the target region is referred to as a "second target region 43." The monochromatic component image is an example of a visible light image. Note that the monochromatic component extracted from the first visible light image by the third calculation unit 60 is not limited to the red component, and for example, a monochromatic component image may be generated from the first visible light image by using only the green component or only the blue component as the monochromatic component.

The third calculation unit 60 calculates a representative value of brightness for the second target region 43, which is an area in which the flame 131 is mainly reflected in the monochromatic component image. Here, the second target region 43 that is the object of calculation by the third calculation unit 60 is equally divided into multiple regions. In FIG. 4, an example is shown in which the second target region 43 is divided into three regions in the width direction (direction perpendicular to the conveying direction W1) of the furnace body 108. Note that the second target region 43 is not limited to being divided into three equal regions, and may be divided into two or four or more regions. Hereinafter, the second target region 43 shown in FIG. 4 will be referred to as the "left region 43l", the "center region 43c", and the "right region 43r" in order from left to right. The third calculation unit 60 calculates the median value of the brightness of each of the left region 43l, the center region 43c, and the right region 43r in the second target region 43 in the monochromatic component image as a representative value. The representative value calculated by the third calculation unit 60 is not limited to the median, and may be a statistical value such as an average value. In other words, the method of calculating the representative value by the third calculation unit 60 is not necessarily limited to the above. The third calculation unit 60 sends the representative values of the luminance of the left region 43l, the center region 43c, and the right region 43r in the calculated single-color component image to the determination unit 70.

(Fourth Calculation Unit)
The fourth calculation unit 65 receives a visible light image from the images received from the acquisition unit 40, and calculates a fourth feature amount based on the received multiple visible light images. In this embodiment, the fourth feature amount is a representative value of luminance based on the multiple visible light images received by the fourth calculation unit 65. Specifically, the fourth calculation unit 65 calculates a representative value of luminance based on multiple visible light images captured within a time it takes for one collapse of the garbage Fg, from among two or more visible light images in a time series that were acquired before the first visible light image used by the third calculation unit 60 to calculate the representative value.

The two or more visible light images used by the fourth calculation unit 65 to calculate the representative value of luminance are visible light images captured at a predetermined time interval from each other. Hereinafter, the time interval existing between two or more visible light images is referred to as the "third time". In this embodiment, the fourth calculation unit 65 calculates the representative value of luminance based on each of the number of visible light images of the number of frames acquired per unit time by the acquisition unit 40. Therefore, the third time is the second predetermined period. The third time is, for example, less than one second long.

In this embodiment, the two or more visible light images are images captured before the first visible light image for a fourth time period that is at least twice the third time period. The fourth calculation unit 65 calculates a representative value of luminance based on the two or more visible light images, for example, three or more (four or more) visible light images. In this embodiment, the fourth calculation unit 65 calculates a representative value of luminance based on each of the images of the number of frames acquired per unit time by the acquisition unit 40. Here, when the time required for one collapse of the garbage Fg is S, the second predetermined period is T _ii , and the number of visible light images (number of frames) used in the calculation is N _ii , the following formula (ii) is established.
S/2< _Tii × _Nii ... (ii)
That is, the multiple visible light images are multiple images acquired over a time period longer than at least half the time period S. In this embodiment, the multiple visible light images are acquired over a time period S. Hereinafter, each of the multiple images used by the fourth calculator 65 to calculate the representative value of luminance will be referred to as a "second visible light image." The second visible light image is an example of the second image.

The fourth calculation unit 65 generates a monochromatic component image from the second infrared image and calculates a representative value of the luminance of a specific target region that is a part of the monochromatic component image. The target region that the fourth calculation unit 65 calculates is the same region as the second target region 43 described above. The fourth calculation unit 65 calculates the median value of the luminance of each of the left region 43l, the central region 43c, and the right region 43r of the second target region 43. Furthermore, the fourth calculation unit 65 calculates the median value of the entire left region 43l, the entire central region 43c, and the entire right region 43r as a representative value based on multiple monochromatic component images. Note that the representative value calculated by the fourth calculation unit 65 is not limited to the median value, and may be a statistical quantity such as an average value. In other words, the method of calculating the representative value by the fourth calculation unit 65 is not necessarily limited to the above. The fourth calculation unit 65 sends the representative values of the luminance of each of the multiple left region 43l, the multiple center region 43c, and the multiple right region 43r based on the multiple calculated single-color component images to the determination unit 70.

(Determination unit)
Returning to Fig. 2, the configuration of the determination unit 70 will be described. The determination unit 70 performs various determination processes (described later) based on the image received from the acquisition unit 40 and the feature amounts (e.g., representative values of luminance) received from each of the first calculation unit 50, the second calculation unit 55, the third calculation unit 60, and the fourth calculation unit 65. In this embodiment, the determination unit 70 has, for example, a first determination unit 71, a second determination unit 72, a disturbance determination unit 74, a third determination unit 73, a first collapse determination unit 75, and a second collapse determination unit 76.

(First Determination Unit)
The first determination unit 71 performs a determination regarding collapse based on the representative value of brightness (first feature amount) received from the first calculation unit 50 and the representative value of brightness (second feature amount) received from the second calculation unit 55. Hereinafter, the determination by the first determination unit 71 will be referred to as the "first determination." The first determination unit 71 calculates a feature change amount V1 (absolute value) occurring between the first infrared image that is the subject of the first determination and the multiple second infrared images according to the following formula (iii).
V1=|A _m (t)-(ΣB _m (t _s ))/s| ... (iii)

In the above formula (iii), A _m (t) is a representative value of luminance (first feature amount) calculated by the first calculation unit 50 at time t. ΣB _m (t _s ) is a representative value of luminance (second feature amount) calculated by the second calculation unit 55. _{t s} are multiple times at which each second infrared image is acquired, and one second infrared image corresponds to one time t _s . s is the number of second infrared images, which is the number of frames acquired per unit time.

The first determination unit 71 performs a first determination as to whether the calculated characteristic change amount V1 is equal to or greater than a predetermined first threshold value. When the calculated characteristic change amount V1 is equal to or greater than the first threshold value, the first determination unit 71 determines that there is a possibility of collapse, and when the characteristic change amount V1 is less than the first threshold value, the first determination unit 71 determines that there is no possibility of collapse. In this embodiment, when the characteristic change amount V1 is equal to or greater than the first threshold value, the first determination unit 71 outputs a flag (i=1) indicating that there is collapse. On the other hand, when the characteristic change amount V1 is less than the first threshold value, the first determination unit 71 outputs a flag (i=0) indicating that there is no collapse. Hereinafter, the flag (i=1 or 0) regarding the determination of collapse output by the first determination unit 71 is referred to as a "first collapse determination flag". In other words, the first collapse determination flag indicates the result of the first determination and indicates a value of 1 or 0. The first threshold value is pre-stored in the storage unit 90. The first determination unit 71 performs a first determination by timely referring to the first threshold value stored in the memory unit 90. The first determination unit 71 sends a first collapse determination flag, which is the result of the first determination, to the first collapse determination unit 75 and the second collapse determination unit 76.

Here, FIG. 5 shows the time series change in the characteristic change amount V1 in a specific time period, and the time series change in the first collapse determination flag according to the characteristic change amount V1 in the same time period. The graph shown in FIG. 5 is the result obtained by the inventors' analysis. In FIG. 5(b), the time when the operator (experienced worker) of the incineration equipment 100 visually inspects the inside of the combustion chamber R and determines that the waste Fg has collapsed is indicated by a circle (◯). In addition, the time corresponding to the interval between the time scale lines (dotted and dashed lines) extending up and down in the graph shown in FIG. 5 is equal for all intervals. From the results shown in FIG. 5, it can be seen that the time when the operator of the incineration equipment 100 determines that a collapse has occurred and the time when the first collapse determination flag is raised (the time when i=1) are roughly the same.

(Second Judgment Unit)
The second determination unit 72 makes a determination regarding collapse based on one or more infrared images received from the acquisition unit 40. In this embodiment, the second determination unit 72 makes a determination regarding collapse using one first infrared image (the most recent infrared image) that is the target of calculation by the first calculation unit 50 among the images received from the acquisition unit 40. Note that, when making a determination regarding collapse, the second determination unit 72 may use one image other than the first infrared image among the images received from the acquisition unit 40. Furthermore, when making a determination regarding collapse, the second determination unit 72 may use multiple images, and the first infrared image may be included in the multiple images.

The second judgment unit 72 performs judgment using a trained model that has been trained to output a judgment result regarding the possibility of collapse when the first infrared image is input. Hereinafter, the judgment by the second judgment unit 72 is referred to as the "second judgment", and the trained model used by the second judgment unit 72 for the second judgment is referred to as the "trained model for second judgment 91". The trained model for second judgment 91 is pre-stored in the memory unit 90 (see FIG. 2). The second judgment unit 72 inputs the first infrared image received from the acquisition unit 40 to the trained model for second judgment 91 stored in the memory unit 90, thereby acquiring the output judgment result as the result of the second judgment. The trained model for second judgment 91 is, for example, a deep learning model (supervised learning model) such as a convolutional neural network (CNN). The second judgment trained model 91 is generated (trained) by repeating a learning step multiple times (e.g., several thousand times) in which an infrared image captured by the imaging device 2 is input and the presence or absence of collapse in the infrared image (correct answer data determined to be correct by a human) is taught. Note that the second judgment trained model 91 may use a recurrent neural network (RNN) or the like instead of a CNN.

Here, FIG. 6 shows image example (a, d) showing that garbage Fg is flying during collapse (with collapse), and image example 1 (b, e) and image example 2 (c, f) showing that there is no collapse (without collapse), divided into cases where the inside of the combustion chamber R is easy to see and cases where it is difficult to see, among the infrared images captured by the imaging device 2. As shown by the two-dot chain line in each image example in FIG. 6, in this embodiment, the second judgment unit 72 inputs an infrared image of a target area, which is a part of the first infrared image, to the second judgment trained model 91. Hereinafter, this target area is referred to as the "third target area 44". The third target area 44 is, for example, most of the area in the upper half of the first infrared image, and is an area in which at least all or most of the garbage Fg accumulated in the feeder 104 is reflected. As shown in the example image (a) in FIG. 6, when the garbage Fg collapses, some (plural) of the collapsed garbage Fg may temporarily scatter (fly away) within the combustion chamber R, making it impossible to visually recognize the layer structure of the garbage Fg accumulated in the feeder 104. The second trained model for judgment 91 is generated in advance by inputting infrared images of the third target region 44 as shown in FIG. 6 into the second trained model for judgment 91 and teaching the presence or absence of collapse (correct answer data) for each infrared image. When a new infrared image is input, the second trained model for judgment 91 that has completed training outputs a numerical value related to the possibility of collapse occurring for that infrared image. Hereinafter, the numerical value output by the second trained model for judgment 91 is referred to as the "judgment score."

The second judgment unit 72 performs a second judgment to judge whether the judgment score acquired from the second judgment trained model 91 is equal to or greater than a predetermined second threshold. When the acquired judgment score is equal to or greater than the second threshold, the second judgment unit 72 judges that there is a possibility of collapse, and when the judgment score is less than the second threshold, the second judgment unit 72 judges that there is no possibility of collapse. In this embodiment, when the judgment score is equal to or greater than the second threshold, the second judgment unit 72 outputs a flag (i = 1) indicating that there is collapse. On the other hand, when the judgment score is less than the second threshold, the second judgment unit 72 outputs a flag (i = 0) indicating that there is no collapse. Hereinafter, the flag (i = 1 or 0) regarding the judgment of collapse output by the second judgment unit 72 is referred to as a "second collapse judgment flag". In other words, the second collapse judgment flag indicates the result of the second judgment and indicates a value of 1 or 0. The second threshold is pre-stored in the memory unit 90. The second determination unit 72 performs the second determination by timely referring to the second threshold value stored in the memory unit 90. The second determination unit 72 sends a second collapse determination flag, which is the result of the second determination, to the first collapse determination unit 75.

FIG. 7 shows the time series changes in the judgment scores for the time periods shown in FIG. 5, and the time series changes in the second collapse judgment flag according to the judgment scores for the same time periods. The graphs shown in FIG. 7 are the results obtained by the inventors' analysis. In FIG. 7(b), the time when the operator of the incineration equipment 100 visually judged that the waste Fg had collapsed by looking inside the combustion chamber R is indicated by a circle (◯). In addition, the time corresponding to the intervals between the time scale lines (dotted lines) extending up and down in the graph shown in FIG. 7 are equal for all intervals. From the results shown in FIG. 7, it can be seen that the time when the operator of the incineration equipment 100 judged that a collapse had occurred and the time when the second collapse judgment flag is raised (the time when i=1) are roughly the same.

(Disturbance determination unit)
The disturbance determination unit 74 determines whether or not there is a disturbance based on a feature amount related to the luminance of one image (e.g., the most recent image) received from the acquisition unit 40. Hereinafter, the determination by the disturbance determination unit 74 is referred to as "disturbance determination". In this embodiment, the disturbance determination unit 74 performs disturbance determination using a first infrared image that is a calculation target of the first calculation unit 50 among the images received from the acquisition unit 40. Note that, when performing disturbance determination, the disturbance determination unit 74 may use an image other than the first infrared image among the images received from the acquisition unit 40. Furthermore, when performing disturbance determination, the disturbance determination unit 74 may use a plurality of images, and the first infrared image may be included in the plurality of images. Furthermore, when receiving an infrared image from the acquisition unit 40, the disturbance determination unit 74 may convert the RAW data (16 bits) into, for example, BMP data (8 bits).

The disturbance determination unit 74 calculates, for example, a feature value related to the overall brightness of the first target region 42 in the first infrared image, a feature value related to the brightness of each of the first region 42a to the ninth region 42i in the first target region 42, and a feature value related to the brightness of a plurality of target regions that are part of the first infrared image and are different from the first target region 42. Hereinafter, the plurality of target regions different from the first target region 42 are referred to as "fourth target region 45". As shown in FIG. 8, the fourth target region 45 is, for example, divided into three locations in the first infrared image, and these three fourth target regions 45 are independent of each other. "Independent" here means that the fourth target regions 45 are separated from each other in the first infrared image. One of the fourth target regions 45 (45a in FIG. 8) is a portion above the first target region 42 in the first infrared image, and the ceiling of the furnace body 108, for example, is reflected in the fourth target region 45. One of the fourth target regions 45 (45b in FIG. 8) is a portion to the right of the first target region 42 in the first infrared image, and the side wall of the furnace body 108, for example, is reflected in the fourth target region 45. One of the fourth target regions 45 (45c in FIG. 8) is a portion below the first target region 42 in the first infrared image, and the fourth target region 45 reflects, for example, the waste Fg that is post-combusted in the post-combustion region 132. Therefore, the waste Fg accumulated in the feeder 104 is hardly reflected in the fourth target region 45. In this embodiment, each of the multiple fourth target regions 45 is made to be a different size.

The disturbance determination unit 74 calculates two types of statistics as features for the entire first target region 42, the first region 42a to the ninth region 42i, and each of the multiple fourth target regions 45. In this embodiment, the two types of statistics are the average luminance and the standard deviation of luminance. Hereinafter, the feature related to the luminance of the entire first target region 42 will be referred to as "feature A", the feature related to the luminance in each of the first region 42a to the ninth region 42i will be referred to as "feature B", and the feature related to the luminance of each of the multiple fourth target regions 45 (45a, 45b, 45c) will be referred to as "feature C". In this embodiment, the disturbance determination unit 74 determines whether the standard deviation of each calculated feature (feature A to feature C) is included in the first range, which is a predetermined numerical range. The disturbance determination unit 74 determines that the first infrared image is a high deviation image when, for example, one or more of the standard deviations of the feature amount A, the standard deviations of the feature amount B, and the standard deviations of the feature amount C are not included in the first range (outside the first range). On the other hand, the disturbance determination unit 74 determines that the first infrared image is a low deviation image when all of the standard deviations of the feature amount A, the standard deviations of the feature amount B, and the standard deviations of the feature amount C are included in the first range. Note that the disturbance determination unit 74 may determine that the first infrared image is a high deviation image when two or more or all of the standard deviations of the feature amount A, the standard deviations of the feature amount B, and the standard deviations of the feature amount C are not included in the first range. The first range is stored in advance in the storage unit 90. The disturbance determination unit 74 performs the determination of the first infrared image by referring to the first range stored in the storage unit 90 at appropriate times.

When the disturbance determination unit 74 receives the above feature values, it performs a determination using a trained model that has been trained to output a determination result indicating the presence or absence of a disturbance. Hereinafter, the trained model used by the disturbance determination unit 74 is referred to as the "trained model for disturbance determination 92". The trained model for disturbance determination 92 is pre-stored in the memory unit 90 (see FIG. 2). The disturbance determination unit 74 inputs feature values A, B, and C to the trained model for disturbance determination 92 stored in the memory unit 90, thereby acquiring the output determination result as the result of the disturbance determination. In this embodiment, the trained model for disturbance determination 92 has a high deviation model and a low deviation model. When the disturbance determination unit 74 determines that the first infrared image is a high deviation image, it inputs feature values (feature values A, B, and C) only to the high deviation model, and acquires the determination result output from the high deviation model as the result of the disturbance determination. On the other hand, if the disturbance determination unit 74 determines that the first infrared image is a low-deviation image, it inputs the feature amount only to the low-deviation model and obtains the determination result output from the low-deviation model as the disturbance determination result.

In this embodiment, the high deviation model and the low deviation model are supervised learning models such as a Support Vector Machine (SVM). Both the high deviation model and the low deviation model are generated (learned) by repeatedly repeating a learning step in which the above-mentioned features are input and the presence or absence of collapse in the image (correct answer data determined to be correct by a human) is instructed.

Here, FIG. 9 shows an example of a low deviation image (a) and an example of a high deviation image (b) among the infrared images captured by the imaging device 2. FIG. 10 shows examples of infrared images in a list format corresponding to the magnitude of the deviation in brightness. As shown in FIG. 9 and FIG. 10, it can be seen that the larger or smaller the deviation in brightness, which is a feature, the more difficult it is to grasp the state inside the combustion chamber R. The feature values based on multiple infrared images as shown in FIG. 9 and FIG. 10 are input to the trained model for disturbance determination 92, and the presence or absence of disturbance (correct answer data) for each infrared image is taught, thereby generating the trained model for disturbance determination 92 in advance. When a feature value based on a new infrared image is input, the trained model for disturbance determination 92 that has completed learning outputs binary data (e.g., 0 and 1) indicating the presence or absence of collapse for the infrared image. The disturbance determination unit 74 determines the presence or absence of a disturbance as a disturbance determination by outputting a flag corresponding to the value of the binary data acquired from the trained model for disturbance determination 92. Hereinafter, the flag related to the disturbance determination output by the disturbance determination unit 74 will be referred to as the "disturbance determination flag." Therefore, the disturbance determination flag indicates the result of the disturbance determination.

FIG. 11 shows the time series of the judgment results regarding the presence or absence of disturbance by the operator of the incineration facility 100 in a specific time period, and the time series of the disturbance judgment flag in the same time period. The graph shown in FIG. 11 is the result obtained by the analysis by the inventors. Note that the time periods T ₀ -T ₉ shown in FIG. 11 are the same time periods as the time periods T ₀ -T ₉ shown in FIG. 5 and FIG. 7. In addition, the time corresponding to the intervals between the time scale lines (dotted lines) extending up and down in the graph shown in FIG. 11 are equal for all intervals. From the results shown in FIG. 11, it can be understood that the time when the operator of the incineration facility 100 visually judges the presence of a disturbance inside the combustion chamber R and the time when the disturbance judgment flag is raised are almost the same.

(Third Judgment Unit)
The third determination unit 73 performs a determination regarding collapse based on one or more images received from the acquisition unit 40. In this embodiment, the third determination unit 73 receives a visible light image from the images received from the acquisition unit 40, and calculates a representative value of luminance based on the received visible light image. The third determination unit 73 performs a determination regarding collapse based on the representative value of luminance (third feature amount) received from the third calculation unit 60 and the representative value of luminance (fourth feature amount) received from the fourth calculation unit 65. Hereinafter, the determination by the third determination unit 73 is referred to as a "third determination". The third determination unit 73 calculates, for each region (left region 43l, center region 43c, right region 43r), a feature change amount V2 (absolute value) occurring between a monochromatic component image based on a first visible light image to be subjected to the third determination and a monochromatic component image based on a plurality of second visible light images, according to the following formula (iv). V2=|C _m (t)-(ΣD _m (t _u ))/u| ... (iv)

In the above formula (iv), C _m (t) is a representative value of luminance (third feature amount) calculated by the third calculation unit 60 at time t. ΣD _m (t _u ) is a representative value of luminance (fourth feature amount) calculated by the fourth calculation unit 65. _{t u} are multiple times at which each second visible light image (monochrome component image) was acquired, and one second visible light image corresponds to one time t _u . u is the number of second visible light images (monochrome component images), which is the number of frames acquired per unit time.

The third determination unit 73 performs a third determination for each region by determining whether the calculated feature change amount V2 is equal to or greater than a predetermined third threshold. The third determination unit 73 determines that there is a possibility of collapse when the feature change amount V2 of one or more regions among the calculated feature change amount V2 for each region is equal to or greater than the third threshold, and determines that there is no possibility of collapse when the feature change amount V2 for all regions is less than the third threshold. In this embodiment, the third determination unit 73 outputs a flag (i = 1) indicating the presence of collapse when the feature change amount V2 is equal to or greater than the third threshold. On the other hand, the third determination unit 73 outputs a flag (i = 0) indicating the absence of collapse when the feature change amount V2 is less than the third threshold. Hereinafter, the flag (i = 1 or 0) regarding the determination of collapse output by the third determination unit 73 is referred to as a "third collapse determination flag". In other words, the third collapse determination flag indicates the result of the third determination and indicates a value of 1 or 0. The third threshold is pre-stored in the storage unit 90. The third threshold value may be a different value for each region. The third determination unit 73 performs the third determination by referring to the third threshold value stored in the storage unit 90 at appropriate times. The third determination unit 73 sends a third collapse determination flag, which is the result of the third determination, to the first collapse determination unit 75.

Here, FIG. 12 shows the time series changes in the characteristic change amount V2 of each area (left area 43l, center area 43c, right area 43r) in a specific time period, and the time series changes in the third collapse determination flag according to the characteristic change amount V2 in the same time period. The graph shown in FIG. 12 is the result obtained by the analysis by the inventors. In FIG. 12(d), the time when the operator of the incineration equipment 100 visually judged that the waste Fg had collapsed by looking inside the combustion chamber R is indicated by a circle (◯). In addition, the time corresponding to the interval between the time scale lines (dotted lines) extending up and down in the graph shown in FIG. 12 is equal for all intervals. From the results shown in FIG. 12, it can be seen that the time when the operator of the incineration equipment 100 judged that there had been a collapse and the time when the third collapse determination flag is raised (the time when i=1) are roughly the same.

(First collapse determination unit)
The first collapse determination unit 75 determines the scale of the collapse based on the result of the first determination received from the first determination unit 71, the result of the second determination received from the second determination unit 72, and the result of the third determination received from the third determination unit 73. In this embodiment, the first collapse determination unit 75 determines the presence or absence of a second-scale collapse, which is larger in scale than the first-scale collapse, based on the first collapse determination flag, the second collapse determination flag, and the third collapse determination flag. Specifically, the first collapse determination unit 75 determines that a second-scale collapse has occurred when the sum of the value of the first collapse determination flag, the value of the second collapse determination flag, and the value of the third collapse determination flag is equal to or greater than the determination threshold value. That is, the first collapse determination unit 75 detects a collapse of the second scale. When the first collapse determination unit 75 detects a collapse of the second scale, it outputs a flag indicating the presence of a collapse. On the other hand, the first collapse determination unit 75 determines that there is no collapse of the second scale when the sum of the value of the first collapse determination flag, the value of the second collapse determination flag, and the value of the third collapse determination flag is less than the determination threshold value. That is, the first collapse determination unit 75 detects that there is no collapse of the second scale. When the first collapse determination unit 75 detects that there is no collapse of the second scale, it outputs a flag indicating that there is no collapse of the second scale. Hereinafter, the flag output by the first collapse determination unit 75 is referred to as a "second scale collapse detection flag". The determination threshold value is stored in advance in the storage unit 90. In this embodiment, the determination threshold value is an integer, for example, 2 is adopted. That is, in this embodiment, the first collapse determination unit 75 determines that there is a collapse of the second scale when two-thirds or more of the determination results of the first determination result, the second determination result, and the third determination result indicate the possibility of collapse. The first collapse determination unit 75 determines whether or not a collapse of the second scale has occurred by timely referring to the determination threshold value stored in the memory unit 90. The first collapse determination unit 75 sends a second-scale collapse detection flag to the second collapse determination unit 76 and the control unit 80.

(Second collapse determination unit)
When the first collapse determination unit 75 determines that there is no collapse of the second scale, the second collapse determination unit 76 determines the presence or absence of a collapse of the first scale based on a change in brightness of the multiple images received from the acquisition unit 40. In this embodiment, the second collapse determination unit 76 determines the presence or absence of a collapse of the first scale based on the first collapse determination flag received from the first determination unit 71. Specifically, when the first collapse determination flag indicates that there is a collapse (i=1), the second collapse determination unit 76 determines that there is a collapse of the first scale. That is, the second collapse determination unit 76 detects a collapse of the first scale. When the second collapse determination unit 76 detects a collapse of the first scale, it outputs a flag indicating that there is a collapse. On the other hand, when the first collapse determination flag indicates that there is no collapse (i=0), the second collapse determination unit 76 determines that there is no collapse. That is, the second collapse determination unit 76 detects that there is no collapse. When the second collapse determination unit 76 detects a collapse of the first scale, it outputs a flag indicating no collapse. Hereinafter, the flag output by the second collapse determination unit 76 is referred to as the "first scale collapse detection flag." In addition, below, when there is no need to distinguish between the "first scale collapse detection flag" and the above-mentioned "second scale collapse detection flag," they will simply be referred to as the "collapse detection flag." The second collapse determination unit 76 sends the first scale collapse detection flag to the control unit 80.

(Configuration of the control unit)
The control unit 80 controls the plurality of control target devices S based on the collapse detection flags received from the first collapse determination unit 75 and the second collapse determination unit 76 (see FIG. 1 and FIG. 2). In this embodiment, when the control unit 80 receives the second-scale collapse detection flag indicating the presence of collapse from the first collapse determination unit 75, the control unit 80 controls one or more of the plurality of control target devices S, for example, so that the concentration of unburned gas in the combustion chamber R is reduced. On the other hand, when the control unit 80 receives the first-scale collapse detection flag from the second collapse determination unit 76, the control unit 80 controls one or more of the plurality of control target devices S, for example, so that the control target devices S are operated at rated speed. The control unit 80 transmits, for example, signals indicating the increase or decrease in the moving speed of the push arm 124, the moving speed of the grate 126, the increase or decrease in the number of revolutions of the blower 138, the valve opening degree of the first flow rate control valve 140, and the valve opening degree of the second flow rate control valve 142 to each control target device S. In addition, when the control unit 80 receives a first-scale collapse detection flag from the second collapse determination unit 76, it may control one or more of the multiple control target devices S so that the concentration of unburned gas in the combustion chamber R is reduced to a smaller extent than when the control unit 80 receives a second-scale collapse detection flag.

(Operation of information processing device)
Next, an example of the operation of the information processing device 4 in this embodiment will be described with reference to Fig. 13. However, the order of the processes described below is not limited to the following example, and may be changed as appropriate.

The acquisition unit 40 acquires an infrared image from the imaging device 2 (step S1). The acquisition unit 40 also acquires a visible light image from the imaging device 2 (step S11). Following the process of step S1, the first calculation unit 50 calculates a representative value of brightness based on the first infrared image acquired in step S1. The second calculation unit 55 also calculates a representative value of brightness based on the second infrared image acquired in step S1 (step S2). Next, the disturbance determination unit 74 performs a disturbance determination based on the first infrared image acquired in step S1 (step S3). If the disturbance determination unit 74 determines that there is a disturbance (step S3: YES), the process returns to step S1. On the other hand, if the disturbance determination unit 74 determines that there is no disturbance (step S3: NO), the first determination unit 71 performs a first determination based on the representative value of brightness calculated in step S2 (step S4), and the second determination unit 72 performs a second determination based on the representative value of brightness calculated in step S2 (step S5).

Following the processing of step S11 above, the third calculation unit 60 calculates a representative value of luminance based on the first visible light image acquired in step S11. The fourth calculation unit 65 calculates a representative value of luminance based on the second visible light image acquired in step S1 (step S12). Next, the third judgment unit 73 performs a third judgment based on the representative value of luminance calculated in step S12 (step S13).

Following the processing of steps S4, S5, and S13 above, the first collapse determination unit 75 determines whether or not a second-scale collapse has occurred based on the results of the determination in steps S4, S5, and S6 (step S6). That is, in step S6, the first collapse determination unit 75 determines whether or not the total value of the first collapse determination flag, the second collapse determination flag, and the third collapse determination flag is equal to or greater than the determination threshold value. If the first collapse determination unit 75 determines that a second-scale collapse has occurred (step S6: YES), it detects the second-scale collapse (step S7). When the processing of step S7 is completed, the processing returns to step S1.

On the other hand, if the first collapse determination unit 75 determines that there is no collapse of the second scale (step S6: NO), the second collapse determination unit 76 determines whether there is a change in the representative brightness value (step S8). That is, in step S8, the second collapse determination unit 76 determines whether there is a collapse of the first scale based on the first collapse determination flag. If the second collapse determination unit 76 determines that there is a collapse of the first scale (step S8: YES), it detects the collapse of the first scale (step S9). When the processing of step S9 is completed, the processing returns to step S1. On the other hand, if the second collapse determination unit 76 determines that there is no collapse of the first scale (step S8: NO), it detects that there is no collapse (step S10). When the processing of step S10 is completed, the processing returns to step S1.

The operation of the information processing device 4 described above is repeatedly executed during the operation of the incineration facility 100.

(Action and Effects)
It may be difficult to properly detect the collapse of the garbage Fg because complex phenomena occur in the combustion chamber R. For example, one of the reasons may be that ash 135 or the like may fly up in the combustion chamber R at an unintended timing, and the flying ash 135 may be reflected for an instant in the image.

In this embodiment, the presence or absence of collapse of the garbage Fg is determined based on a representative value of brightness based on the first infrared image and a representative value of brightness based on multiple second infrared images captured within the time it takes for one collapse of the garbage Fg, among multiple infrared images in a time series acquired before the first infrared image. Therefore, the presence or absence of collapse of the garbage Fg can be determined with higher accuracy compared to, for example, a case in which the presence or absence of collapse is determined based on the brightness of each image acquired at two timings. In other words, the detection accuracy regarding the collapse of the garbage Fg can be improved. As a result, the combustion state of the garbage Fg in the combustion chamber R can be accurately grasped.

FIG. 14 is a diagram showing an example of the results of the time series of the collapse detection flags output by the first collapse determination unit 75 and the second collapse determination unit 76 according to this embodiment. That is, FIG. 14 shows the final determination result (detection result) by the information processing device 4 in this embodiment. The graph shown in FIG. 14 is a result obtained by the analysis by the inventors. In FIG. 14, the time when the operator of the incineration equipment 100 visually checks the inside of the combustion chamber R and determines that the first scale of garbage Fg has collapsed is shown by a triangle (△), and the time when the second scale of garbage Fg has collapsed is shown by a circle (◯). In addition, the time corresponding to the interval between the time scale lines (dotted lines) extending up and down in the graph shown in FIG. 14 is equal for all intervals. From the results shown in Figure 14, although there are times (between time T _-18 and time T _-17 ) when the information processing device 4 may have overly detected the collapse of the waste Fg, it can be seen that the time when the operator of the incineration equipment 100 determined that a collapse had occurred almost coincides with the time when the collapse determination flag is raised.

<Second embodiment of incineration facility>
Next, a second embodiment of the incineration facility 100 according to the present disclosure will be described. In the second embodiment described below, components common to the first embodiment will be denoted by the same reference numerals in the drawings and will not be described. In the second embodiment, the configuration of the second collapse determination unit 76 of the collapse detection unit 41 is different from the second collapse determination unit 76 described in the first embodiment.

In this embodiment, the second collapse determination unit 76 calculates the similarity between the image acquired by the acquisition unit 40 and one or more images previously acquired by the acquisition unit 40, and determines that there is no collapse if the calculated similarity does not satisfy a predetermined condition. Below, the determination of the presence or absence of a collapse by the second collapse determination unit 76 will be described using as an example a case in which the second determination unit 72 calculates the similarity between the first infrared image at time t and multiple infrared images acquired in the past prior to the first infrared image.

As shown in FIG. 15, the second collapse determination unit 76 binarizes the luminance included in a specific target area (b), which is a part of the infrared image (a) received from the acquisition unit 40, into data of 0 and 1 (0/1 conversion). Hereinafter, the target area to be binarized by the second collapse determination unit 76 is referred to as the "fifth target area 46". The fifth target area 46 is, for example, an area in the upper half of the infrared image in which a part (most part) of the garbage Fg accumulated in the feeder 104 is reflected. The second collapse determination unit 76 binarizes the luminance included in the fifth target area 46, for example, based on a predetermined luminance threshold value. The luminance threshold value is stored in advance in the storage unit 90. The second collapse determination unit 76 binarizes the fifth target area 46 by referring to the luminance threshold value stored in the storage unit 90 at appropriate times. Hereinafter, the fifth target area 46 binarized by the second collapse determination unit 76 is referred to as the "binarized data 46'".

FIG. 16 is a diagram for conceptually explaining a method of calculating the similarity calculated by the second collapse determination unit 76. As shown in FIG. 16, the second collapse determination unit 76 acquires the difference between the binarized data 46' based on the first infrared image at time t and the binarized data 46' based on each of a plurality of infrared images acquired in the past before the first infrared image (X ₁ , X ₂ , ..., X _y-1 , X _y shown in FIG. 16), and calculates the average value of the acquired differences (ΣX/y shown in FIG. 16) as the similarity. The binarized data 46' based on each of the plurality of infrared images here means, for example, the binarized data 46' based on the infrared image acquired at time t-1, the binarized data 46' based on the infrared image acquired at time t-2, ..., the binarized data 46' based on the infrared image acquired at time t-(y-1), and the binarized data 46' based on the infrared image acquired at time t-y. y is, for example, an integer, and a value of 2 or more (such as 5) is adopted.

The second collapse determination unit 76 determines whether the calculated similarity satisfies a predetermined condition. In this embodiment, the second collapse determination unit 76 determines whether the similarity is equal to or greater than a predetermined similarity threshold. In this case, the second collapse determination unit 76 determines that the predetermined condition is satisfied when the similarity is equal to or greater than the similarity threshold, and that the predetermined condition is not satisfied when the similarity is less than the similarity threshold. The similarity threshold is pre-stored in the memory unit 90. The second collapse determination unit 76 determines whether the calculated similarity satisfies a predetermined condition by referring to the similarity threshold stored in the memory unit 90 at appropriate times. The second collapse determination unit 76 determines that a first-scale collapse has occurred when the similarity does not satisfy the predetermined condition. That is, the second collapse determination unit 76 detects a first-scale collapse. When the second collapse determination unit 76 detects a first-scale collapse, it outputs a first-scale collapse detection flag indicating the presence of a collapse. On the other hand, the second collapse determination unit 76 determines that there is no collapse when the similarity satisfies a predetermined condition. In other words, the second collapse determination unit 76 detects that there is no collapse. When the second collapse determination unit 76 detects a collapse of the first scale, it outputs a collapse detection flag indicating that there is no collapse.

Next, an example of the operation of the information processing device 4 will be described with reference to FIG. 17. However, the order of the processes described below is not limited to the following example, and may be changed as appropriate. Also, descriptions of parts that overlap with the operation of the information processing device 4 described using FIG. 13 will be omitted.

If the second collapse determination unit 76 determines that there is a change in the representative brightness value (step S8: YES), it calculates the similarity and determines whether the calculated similarity satisfies a predetermined condition (step S20). On the other hand, if the second collapse determination unit 76 determines that there is no change in the representative brightness value (step S8: NO), it detects that there is no collapse (step S10). If the similarity does not satisfy the predetermined condition (step S20: NO), the second collapse determination unit 76 detects a collapse of the first scale (step S9). On the other hand, if the similarity satisfies the predetermined condition (step S20: YES), the second collapse determination unit 76 executes the processing of step S10.

FIG. 18 is a diagram showing an example of the results of the time series of the collapse detection flags output by the first collapse determination unit 75 and the second collapse determination unit 76 described above. That is, FIG. 18 shows the final determination result (detection result) by the information processing device 4. The graph shown in FIG. 18 is a result obtained by the analysis by the inventors. In FIG. 18, the time when the operator of the incineration equipment 100 visually checks the inside of the combustion chamber R and determines that the first scale of garbage Fg has collapsed is shown by a triangle (△), and the time when the second scale of garbage Fg has collapsed is shown by a circle (◯). In addition, the time corresponding to the interval between the time scale lines (dotted and dashed lines) extending up and down in the graph shown in FIG. 18 is equal for all intervals. In the results shown in FIG. 18, there is no time (between time T _-18 and time T _-17 in FIG. 14) during which the information processing device 4 may have excessively detected the collapse of the garbage Fg, as compared with the results shown in FIG. 14. It is also understood that the time when the operator of the incineration facility 100 determines that a collapse has occurred coincides with the time when the collapse determination flag is raised.

<Third embodiment of incineration equipment>
Next, a third embodiment of the incineration facility 100 according to the present disclosure will be described. In the third embodiment described below, the configurations common to the first embodiment are denoted by the same reference numerals in the drawings and will not be described. In the third embodiment, the configurations of the collapse detection unit 41 and the storage unit 90 are different from those of the first embodiment.

(Configuration of collapse detection unit)
19 , the collapse detection unit 41 in this embodiment has a first calculation unit 50, a second calculation unit 55, a fifth calculation unit 66, a sixth calculation unit 67, and a judgment unit 70. The judgment unit 70 has a disturbance judgment unit 74, a fourth judgment unit 77, a fifth judgment unit 78, a third collapse judgment unit 79, and a fourth collapse judgment unit 81.

(First Calculation Unit)
20, in this embodiment, the first target region 42 that is the target of calculation by the first calculation unit 50 is divided into 24 small regions with equal areas partitioned in a 6 × 4 matrix. Note that the first target region 42 is not limited to being divided equally into 24 small regions in a matrix.

The first calculation unit 50, for example, calculates the average brightness value of each small region in the first target region 42 in the first infrared image. Note that the representative value calculated by the first calculation unit 50 is not limited to the average brightness value of each small region, and may be a statistical value such as the median.

(Fifth Calculation Unit)
When the disturbance determination unit 74 determines that there is no disturbance, the fifth calculation unit 66 calculates a fifth feature based on the representative value of the luminance (first feature) received from the first calculation unit 50 and the representative value of the luminance (second feature) received from the second calculation unit 55. In this embodiment, the fifth feature is a sum of luminance changes obtained by time-differentiating the average value of the luminance of each of the 24 small regions in the first target region 42 in the first infrared image. Note that the sum calculated by the fifth calculation unit 66 is not limited to the sum of the luminance changes of each of the small regions, and may be, for example, the sum of a statistical quantity such as a median. The fifth calculation unit 66 inputs the sum of the luminance changes of each of the small regions in the first target region 42 in the first infrared image to the third collapse determination unit 79.

(Sixth Calculation Unit)
The sixth calculation unit 67 performs unsupervised learning using an autoencoder algorithm, and calculates a sixth feature using dimension-reduced information placed in the hidden layer of the learned autoencoder. In this embodiment, the autoencoder 96 is stored in advance in the storage unit 90 (see FIG. 19). The sixth feature is infrared image information in which a plurality of consecutive infrared images acquired by the imaging device 2 are encoded by the autoencoder 96 and the number of dimensions is reduced. The imaging device 2 acquires one frame of infrared image every 0.1 seconds. The sixth calculation unit 67 inputs a plurality of consecutive infrared images acquired by the imaging device 2 at intervals of 0.1 seconds to the autoencoder 96. In the autoencoder 96, the plurality of infrared images are compressed (encoded) in the process of flowing from the input layer to the hidden layer, and the number of dimensions is reduced to 512 dimensions. After that, the infrared image with the reduced number of dimensions is restored (decoded) to the original information in the process of flowing from the hidden layer to the output layer. If the infrared image placed in the input layer can be restored from the infrared image flowing from the hidden layer to the output layer, the learning data of the infrared image does not contain abnormal data such as disturbance images, and it is considered that correct learning has been performed. If correct learning is performed, the infrared image information with reduced dimensions has reduced noise in the image. Note that the interval at which the imaging device 2 captures infrared images is not limited to 0.1 seconds. Also, the dimension reduction performed in the autoencoder 96 is not limited to 512 dimensions.

When the third collapse determination unit 79 determines that there is a collapse in the furnace as described below, the sixth calculation unit 67 packages 40 consecutive frames of infrared image information, from which the number of dimensions has been reduced and noise in the images has been removed, that have been acquired over a total of four seconds, two seconds before and two seconds after the point in time when a human judges that a waste collapse has occurred, from among a plurality of consecutive infrared image information at 0.1 second intervals, from which the number of dimensions has been reduced and noise in the images has been removed, and inputs the packaged infrared image information into a long short-term memory (LSTM) network included in the fourth collapse determination unit 81. The long short-term memory network is a type of RNN that learns and predicts (regression and classification) time-series data, and the fourth collapse determination unit 81 makes a judgment using a trained model that has been trained to output a judgment result regarding the possibility of a collapse when the packaged infrared image information is input to the long short-term memory network. Hereinafter, the trained model used for the judgment by the fourth collapse determination unit 81 is referred to as the "trained model for collapse judgment 97". The trained model 97 for determining collapse is stored in advance in the storage unit 90 (see FIG. 19). The number of information frames to be packaged is not limited to 40. It is sufficient to observe how long it takes for the series of events to progress, from the time the garbage peels off the garbage surface inside the furnace, the time it lands on the furnace floor, the time it floats inside the furnace, and the time it falls from the garbage that has floated inside the furnace, and to the time it can be increased or decreased as appropriate, taking into account the characteristics of the incinerator, etc. Also, since a human's judgment that a garbage collapse has occurred includes uncertainty, the following five frames, including the infrared image information at the time when the human judges that a garbage collapse has occurred, may be used as the training data for the occurrence of the collapse.

(Fourth Judgment Unit)
The fourth determination unit 77 makes a determination regarding collapse based on one or more infrared images received from the acquisition unit 40. In this embodiment, the fourth determination unit 77 makes a determination regarding collapse using one first infrared image (the most recent infrared image) that is the target of calculation by the first calculation unit 50 among the images received from the acquisition unit 40. Note that, when making a determination regarding collapse, the fourth determination unit 77 may use one image other than the first infrared image among the images received from the acquisition unit 40. Furthermore, when making a determination regarding collapse, the fourth determination unit 77 may use multiple images, and the first infrared image may be included in the multiple images.

The fourth judgment unit 77 makes a judgment using a trained model that has been trained to output a judgment result regarding the possibility of a collapse when the first infrared image is input. Hereinafter, the judgment by the fourth judgment unit 77 is referred to as the "fourth judgment", and the trained model that the fourth judgment unit 77 uses for the fourth judgment is referred to as the "trained model for fourth judgment 94". The trained model for fourth judgment 94 is pre-stored in the memory unit 90 (see FIG. 19). The fourth judgment unit 77 inputs the first infrared image received from the acquisition unit 40 to the trained model for fourth judgment 94 stored in the memory unit 90, thereby acquiring the output judgment result as the result of the fourth judgment. The trained model for fourth judgment 94 is, for example, a deep learning model (supervised learning model) such as a convolutional neural network (CNN). The fourth judgment trained model 94 is generated (trained) by repeating a learning step multiple times in which an infrared image captured by the imaging device 2 is input and the presence or absence of collapse in the infrared image (correct answer data determined to be correct by a human) is taught. Note that the fourth judgment trained model 94 may use a recurrent neural network (RNN) instead of a CNN.

In the infrared images captured by the imaging device 2, image examples (a, d) showing garbage Fg flying during collapse (collapse), image examples (b, e) showing ash flying without the effect of collapse (no collapse), and image examples (c, f, including water vapor retention) showing other states (no collapse) are shown in FIG. 21. In this embodiment, when an infrared image of a target area that is a part of the first infrared image is input to the fourth trained model for judgment 94, the fourth judgment unit 77 classifies the presence or absence of collapse for the infrared image captured by the imaging device 2 based on the above three classified image examples. The fourth trained model for judgment 94 is generated in advance by being taught the presence or absence of collapse for the infrared image (correct answer data). When a new infrared image is input, the fourth trained model for judgment 94 that has completed learning outputs a numerical value related to the possibility of collapse occurring for the infrared image as a "judgment score".

The fourth judgment unit 77 judges whether the judgment score acquired from the fourth judgment trained model 94 is equal to or greater than a predetermined fourth threshold (fourth judgment). When the acquired judgment score is equal to or greater than the fourth threshold, the fourth judgment unit 77 judges that there is a possibility of collapse, and when the judgment score is less than the fourth threshold, it judges that there is no possibility of collapse. In this embodiment, when the judgment score is equal to or greater than the fourth threshold, the fourth judgment unit 77 outputs a flag (i = 1) indicating that there is collapse. On the other hand, when the judgment score is less than the fourth threshold, the fourth judgment unit 77 outputs a flag (i = 0) indicating that there is no collapse. Hereinafter, the flag (i = 1 or 0) regarding the judgment of collapse output by the fourth judgment unit 77 is referred to as the "fourth collapse judgment flag". In other words, the fourth collapse judgment flag indicates the result of the fourth judgment and indicates a value of 1 or 0. The fourth threshold is pre-stored in the memory unit 90. The fourth determination unit 77 performs the fourth determination by timely referring to the fourth threshold value stored in the memory unit 90. The fourth determination unit 77 inputs a fourth collapse determination flag, which is the result of the fourth determination, to the third collapse determination unit 79.

(Fifth Judgment Unit)
The fifth determination unit 78 makes a determination regarding collapse based on the visible light image received from the acquisition unit 40. In this embodiment, the fifth determination unit 78 makes a determination regarding collapse using a first visible light image (the most recent visible light image) among the images received from the acquisition unit 40. Note that, when making the determination regarding collapse, the fifth determination unit 78 may use one image other than the first visible light image among the images received from the acquisition unit 40. Furthermore, when making the determination regarding collapse, the fifth determination unit 78 may use multiple images, and the first visible light image may be included in the multiple images.

The fifth judgment unit 78 performs judgment using a trained model that has been trained to output a judgment result regarding the possibility of collapse when the first visible light image is input. Hereinafter, the judgment by the fifth judgment unit 78 is referred to as the "fifth judgment", and the trained model that the fifth judgment unit 78 uses for the fifth judgment is referred to as the "trained model for fifth judgment 95". The trained model for fifth judgment 95 is stored in advance in the storage unit 90 (see FIG. 19). The fifth judgment unit 78 inputs the first visible light image received from the acquisition unit 40 to the trained model for fifth judgment 95 stored in the storage unit 90, thereby acquiring the output judgment result as the result of the fifth judgment. The trained model for fifth judgment 95 is, for example, a deep learning model (supervised learning model) such as a convolutional neural network (CNN). The fifth trained model for judgment 95 is generated (trained) by repeating a learning step multiple times in which a visible light image captured by the imaging device 2 is input and the presence or absence of collapse in the visible light image (correct answer data determined to be correct by a human) is taught. Note that the fifth trained model for judgment 95 may use a recurrent neural network (RNN) instead of a CNN.

The fifth judgment unit 78 classifies the first visible light image based on example images in which garbage covers flames during a large-scale collapse, and example images showing other conditions (no collapse). That is, a visible light image of a target region that is part of the first visible light image is input to the fifth trained judgment model 95, and the presence or absence of collapse in the first visible light image is classified based on the above two example images. The fifth trained judgment model 95 is generated in advance by being taught the presence or absence of collapse in the visible light image (correct answer data). When a new visible light image is input, the fifth trained judgment model 95 that has completed learning outputs a numerical value related to the possibility of collapse occurring in the visible light image as a "judgment score".

The fifth judgment unit 78 judges whether the judgment score acquired from the fifth judgment trained model 95 is equal to or greater than a predetermined fifth threshold (fifth judgment). The fifth judgment unit 78 judges that there is a possibility of collapse when the acquired judgment score is equal to or greater than the fifth threshold, and judges that there is no possibility of collapse when the judgment score is less than the fifth threshold. In this embodiment, the fifth judgment unit 78 outputs a flag (i = 1) indicating the presence of collapse when the judgment score is equal to or greater than the fifth threshold. On the other hand, the fifth judgment unit 78 outputs a flag (i = 0) indicating the absence of collapse when the judgment score is less than the fifth threshold. Hereinafter, the flag (i = 1 or 0) regarding the judgment of collapse output by the fifth judgment unit 78 is referred to as the "fifth collapse judgment flag". The fifth threshold is stored in advance in the memory unit 90. The fifth judgment unit 78 performs the fifth judgment by referring to the fifth threshold stored in the memory unit 90 at appropriate times. The fifth determination unit 78 inputs the fifth collapse determination flag, which is the result of the fifth determination, to the third collapse determination unit 79.

(Third collapse judgment section)
The third collapse determination unit 79 determines whether or not the waste in the furnace has collapsed based on the sum of the brightness changes of each of the 24 small regions in the first target region 42 in the first infrared image calculated by the fifth calculation unit 66, a fourth collapse determination flag which is the result of the fourth determination performed by the fourth determination unit 77, and a fifth collapse determination flag which is the result of the fifth determination performed by the fifth determination unit 78. Specifically, three patterns are prepared by combining the sum of the brightness changes in the first target region 42, the fourth collapse determination flag, and the fifth collapse determination flag. As shown in FIG. 22, the first pattern is when the sum of the brightness changes in the first target region 42 is 22 or more or −26 or less, and the fourth collapse determination flag input from the fourth determination unit 77 is a flag (i=1) indicating the presence of collapse. The second pattern is when the sum of the amounts of change in luminance of each of the small regions in first target region 42 is equal to or greater than 20 or equal to or less than − (minus) 13, and the fifth collapse determination flag input from fifth determination unit 78 is a flag indicating the presence of collapse (i = 1). The third pattern is when the sum of the amounts of change in luminance of each of the small regions in first target region 42 is equal to or greater than 10 or equal to or less than − (minus) 13, and both the fourth collapse determination flag input from fourth determination unit 77 and the fifth collapse determination flag input from fifth determination unit 78 are flags indicating the presence of collapse (i = 1).

The third collapse determination unit 79 determines that there is collapse in the furnace when the sum of the brightness change amounts of each of the 24 small areas in the first target area 42 in the first infrared image calculated by the fifth calculation unit 66, the fourth collapse determination flag input from the fourth determination unit 77, and the fifth collapse determination flag input from the fifth determination unit 78 correspond to any of the above first to third patterns, and determines that there is no collapse in the furnace when they do not correspond to any of the above first to third patterns. The threshold value of the sum of the brightness change amounts of the infrared images included in the above first to third patterns may be changed depending on the type of image selected. In other words, when the determination result by deep learning of the visible light image indicates the presence of collapse, it can be considered that a large-scale collapse of garbage is occurring in the furnace, so the range of the threshold value of the sum of the brightness change amounts of the infrared images included in the second pattern may be smaller than the range of the threshold value of the sum of the brightness change amounts of the infrared images included in the first pattern. Furthermore, if the results of the deep learning judgment of the infrared image and the deep learning judgment of the visible light image both indicate the presence of collapse, it can be deemed that there is a high probability that the collapse of the waste is occurring inside the furnace. Therefore, the range of the threshold value of the sum of the brightness change amount of the infrared image included in the third pattern may be smaller than the range of the threshold value of the sum of the brightness change amount of the infrared image included in the first and second patterns.

(Fourth collapse judgment section)
The fourth collapse determination unit 81 includes a long-short-term memory network, which is a type of RNN that performs learning and prediction (regression and classification) of time-series data. The fourth collapse determination unit 81 determines whether or not the waste in the furnace has collapsed by using a trained model for collapse determination 97 that has been trained to output a determination result regarding the possibility of collapse when a package of 40 consecutive frames of infrared image information from which noise in the images has been removed is input from the sixth calculation unit 67 to the long-short-term memory network. The fourth collapse determination unit 81 obtains a determination result by the long-short-term memory network by inputting the package of 40 consecutive frames of infrared image information from which noise in the images has been removed input from the sixth calculation unit 67 to the trained model for collapse determination 97 stored in the storage unit 90.

When a package of 40 consecutive frames of infrared image information from which noise has been removed is input from the sixth calculation unit 67 to the long- and short-term memory network, the fourth collapse determination unit 81 trains the long- and short-term memory network on the package and determines the state inside the furnace from the state transitions of the 40 frames of infrared images. That is, when 40 consecutive frames of infrared image information are input to the trained model for collapse determination 97, the trained model for collapse determination 97 is generated in advance by teaching the presence or absence of collapse (correct answer data) for the infrared image information based on the state inside the furnace over time. When a package of infrared image information is input, the trained model for collapse determination 97 that has completed learning outputs a numerical value related to the possibility of collapse occurring for the 40 frames of infrared image information included in the package as a "determination score".

The fourth collapse judgment unit 81 judges whether the judgment score acquired from the trained model for collapse judgment 97 is equal to or greater than a predetermined sixth threshold (sixth judgment). The fourth collapse judgment unit 81 judges that there is a possibility of collapse when the acquired judgment score is equal to or greater than the sixth threshold, and judges that there is no possibility of collapse when the judgment score is less than the sixth threshold. In this embodiment, the fourth collapse judgment unit 81 outputs a flag (i = 1) indicating that there is a collapse when the judgment score is equal to or greater than the sixth threshold. On the other hand, the fourth collapse judgment unit 81 outputs a flag (i = 0) indicating that there is no collapse when the judgment score is less than the sixth threshold. When the fourth collapse judgment unit 81 outputs the flag (i = 1), the judgment unit 70 finally judges that there is a collapse in the furnace, and when the fourth collapse judgment unit 81 outputs the flag (i = 0), the judgment unit 70 finally judges that there is no collapse in the furnace.

(Configuration of the control unit)
The control unit 80 controls the plurality of control target devices S based on the collapse detection flag received from the fourth collapse determination unit 81 (see FIG. 19). In this embodiment, when the control unit 80 receives a collapse detection flag indicating the occurrence of collapse from the fourth collapse determination unit 81, it controls one or more of the plurality of control target devices S so that the control target devices S operate at a rated speed. The control unit 80 transmits, for example, signals indicating the increase or decrease in the moving speed of the push arm 124, the moving speed of the grate 126, the increase or decrease in the number of revolutions of the blower 138, the valve opening degree of the first flow rate control valve 140, and the valve opening degree of the second flow rate control valve 142 to each control target device S. The control unit 80 may control one or more of the plurality of control target devices S so that the concentration of unburned gas in the combustion chamber R is reduced.

(Operation of information processing device)
Next, an example of the operation of the information processing device 4 in this embodiment will be described with reference to Fig. 23. However, the order of the processes described below is not limited to the following example, and may be changed as appropriate.

The acquisition unit 40 acquires an infrared image from the imaging device 2 (step S1). The acquisition unit 40 also acquires a visible light image from the imaging device 2 (step S11). Following the processing of step S1, the first calculation unit 50 calculates a representative value of brightness based on the first infrared image acquired in step S1. The second calculation unit 55 also calculates a representative value of brightness based on the second infrared image acquired in step S1 (step S2). Next, the disturbance determination unit 74 performs a disturbance determination based on the first infrared image acquired in step S1 (step S3). If the disturbance determination unit 74 determines that there is a disturbance (step S3: YES), the processing returns to step S1. On the other hand, if the disturbance determination unit 74 determines that there is no disturbance (step S3: NO), the fifth calculation unit 66 calculates a fifth feature based on the representative value of brightness (first feature) received from the first calculation unit 50 and the representative value of brightness (second feature) received from the second calculation unit 55 (step S21). The fourth determination unit 77 makes a collapse-related determination based on one or more infrared images received from the acquisition unit 40 (step S22). The fifth determination unit 78 also makes a collapse-related determination based on the visible light image received from the acquisition unit 40 (step S23).

Following the processing of steps S21, S22, and S23, the third collapse determination unit 79 determines whether or not the waste inside the furnace has collapsed based on the sum of the brightness changes of each small area in the first target area 42 in the first infrared image calculated in step S21, the result of the determination in step S22, and the result of the determination in step S23 (step S24). If the third collapse determination unit 79 determines that a collapse has occurred (step S24: i = 1), the sixth calculation unit 67 compiles and packages 40 consecutive frames of infrared image information from which noise has been removed over a total of 4 seconds, for 2 seconds before and after the point in time when the human judged that the waste had collapsed, from among multiple consecutive infrared image information at 0.1 second intervals in which the number of dimensions has been reduced and noise in the image has been removed, and inputs this to the long-term and short-term memory network included in the fourth collapse determination unit 81 (step S25). If the third collapse determination unit 79 determines that there is no collapse (step S24: i = 0), it is detected that there is no collapse of waste in the furnace (step S28). When the processing of step S28 is completed, the processing returns to step S1.

Following the processing of step S25, when a package of 40 consecutive frames of infrared image information from which noise has been removed is input from the sixth calculation unit 67 to the long- and short-term memory network, the fourth collapse determination unit 81 uses the trained model for collapse determination 97 to determine whether or not the waste has collapsed inside the furnace (step S26). If the fourth collapse determination unit 81 determines that a collapse has occurred (step S26: i = 1), it detects that there is a collapse of waste inside the furnace (step S27). When the processing of step S27 is completed, the processing returns to step S1. If the fourth collapse determination unit 81 determines that there is no collapse (step S26: i = 0), it detects that there is no collapse (step 28). When the processing of step S28 is completed, the processing returns to step S1.

(Action and Effects)
Even if an image captures a change in the state inside the furnace that does not truly indicate collapse, for example, a state in which black ash moves from bottom to top inside the furnace, if a single frame image is used as the subject of judgment without considering the passage of time, the movement of ash from bottom to top cannot be grasped, and it is recognized as one element indicating the occurrence of collapse, which may cause overdetection. According to this embodiment, the element of time transition is taken into consideration when grasping the state inside the furnace, and information on 40 consecutive frames of infrared images acquired by the acquisition unit 40 is input to a long-short-term memory network, which is one of the trained models with a recursive structure, and the state inside the furnace is determined from the state transition of 40 consecutive frames of infrared images. This makes it possible to correctly grasp the change in the state inside the furnace that cannot be said to indicate collapse as described above, thereby suppressing the occurrence of overdetection.

Other Embodiments
Although the embodiments of the present disclosure have been described in detail above with reference to the drawings, the specific configurations are not limited to those of the embodiments, and additions, omissions, substitutions, and other modifications of the configurations are possible without departing from the gist of the present disclosure.

The above-mentioned first collapse determination unit 75 may determine whether or not a second-scale collapse, which is larger than the first-scale collapse, has occurred based on both the result of the first determination and the result of the second determination. In this case, the first collapse determination unit 75 may determine that a second-scale collapse has occurred when the sum of the value of the first collapse determination flag and the value of the second collapse determination flag is equal to or greater than the determination threshold value.

The collapse detection unit 41 may also determine whether or not a collapse has occurred based on a first input element, which is the first infrared image acquired by the acquisition unit 40, and a second input element, which is two or more second infrared images captured within the time it takes for one collapse of the garbage Fg among a plurality of images in a time series acquired by the acquisition unit 40 before the first infrared image. In this case, the collapse detection unit 41 makes the determination using a trained model 93 (see FIG. 2; hereinafter, referred to as the "trained model for collapse detection") that has been trained to output a determination result regarding the possibility of a collapse when the first input element and the second input element are input. The trained model for collapse detection 93 is stored in advance in the storage unit 90. The collapse detection unit 41 acquires the output determination result by inputting the first input element and the second input element received from the acquisition unit 40 into the trained model for collapse detection 93 stored in the storage unit 90. The trained model for collapse detection 93 is, for example, a deep learning model such as a convolutional neural network. The trained model 93 for collapse detection is generated by repeatedly performing a learning step in which an infrared image as a first input element and a second input element captured by the imaging device 2 are input, and the presence or absence of a collapse is taught for the infrared image.

Furthermore, the images that the first calculation unit 50 and the second calculation unit 55 receive from the acquisition unit 40 are not limited to infrared images. Furthermore, the images that the third calculation unit 60 and the fourth calculation unit 65 receive from the acquisition unit 40 are not limited to visible light images.

In addition, in the embodiment, the incineration facility 100 is a stoker-type waste incinerator, but is not limited to a stoker-type waste incinerator. The incineration facility 100 may be, for example, a kiln stoker furnace, a biomass fluidized bed boiler, a sludge incinerator, etc. Therefore, the collapse detection system 1 described above may be a system that is applied to incineration facilities such as these kiln stoker furnaces, biomass fluidized bed boilers, and sludge incinerators.

FIG. 24 is a hardware configuration diagram showing the configuration of a computer 1100 according to this embodiment. The computer 1100 includes a processor 1110, a main memory 1120, a storage 1130, and an interface 1140.

The information processing device 4 described above is implemented in one or more computers 1100. The operation of each of the above-mentioned processing units is stored in the storage 1130 in the form of a program. The processor 1110 reads the program from the storage 1130, expands it in the main memory 1120, and executes the above-mentioned processing according to the program. The processor 1110 also secures a memory area in the main memory 1120 corresponding to the above-mentioned memory unit 90 according to the program. The program may be for realizing part of the function to be performed by the computer 1100. For example, the program may be for performing a function by combining it with other programs already stored in the storage 1130 or by combining it with other programs implemented in other devices. The computer 1100 may also be provided with a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include Programmable Array Logic (PAL), Generic Array Logic (GAL), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA). In this case, some or all of the functions realized by the processor 1110 may be realized by the integrated circuit.

Examples of storage 1130 include a magnetic disk, a magneto-optical disk, and a semiconductor memory. Storage 1130 may be an internal medium directly connected to the bus of computer 1100, or an external medium connected to computer 1100 via interface 1140 or a communication line. In addition, when this program is distributed to computer 1100 via a communication line, computer 1100 that receives the program may expand the program in main memory 1120 and execute the above-mentioned processing. In the above embodiment, storage 1130 is a non-transient tangible storage medium. The program may also be for realizing part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that realizes the above-mentioned functions in combination with other programs already stored in storage 1130.

<Additional Notes>
The collapse detection system and the collapse detection method described in each embodiment can be understood, for example, as follows.

(1) The collapse detection system 1 according to the first aspect includes an acquisition unit 40 that acquires images of the incinerated material (garbage Fg) that is piled up in the feeder 104 of the incineration equipment 100 and pushed toward the combustion chamber R at a first predetermined period, a first calculation unit 50 that calculates a representative value of brightness based on the first image (first infrared image) acquired by the acquisition unit 40, a second calculation unit 55 that calculates a representative value of brightness based on two or more images (second infrared images) captured within the time it takes for one collapse of the incinerated material among a plurality of images in a time series acquired by the acquisition unit 40 before the first image, and a determination unit 70 that makes a determination regarding the collapse based on the representative value of brightness calculated by the first calculation unit 50 and the representative value of brightness calculated by the second calculation unit 55.

This makes it possible to determine with higher accuracy whether the garbage Fg has collapsed, for example, compared to when determining whether the garbage Fg has collapsed based on the brightness of each image acquired at two different times. As a result, it is possible to accurately grasp the combustion state of the garbage Fg in the combustion chamber R.

(2) A collapse detection system 1 according to a second aspect is the collapse detection system 1 of (1), in which the second calculation unit 55 may calculate an average or median brightness value for the two or more images as a representative brightness value based on the two or more images.

This allows the state of the first infrared image and the state of the second infrared image to be expressed using specific statistics.

(3) A collapse detection system 1 according to a third aspect is a collapse detection system 1 according to (1) or (2), in which the two or more images are images captured at an interval of a first time from each other, and the two or more images may be images captured before the first image for a second time that is at least twice the first time.

This allows the above effects to be realized in more specific settings.

(4) A collapse detection system 1 according to a fourth aspect is a collapse detection system 1 according to any one of (1) to (3), and the determination unit 70 may include a first determination unit 71 that makes a first determination regarding the collapse based on a representative value of brightness calculated by the first calculation unit 50 and a representative value of brightness calculated by the second calculation unit 55, a second determination unit 72 that makes a second determination regarding the collapse based on one or more images acquired by the acquisition unit 40, and a first collapse determination unit 75 that determines the presence or absence of a second-scale collapse, which is larger in scale than the first-scale collapse, based on the result of the first determination and the result of the second determination.

This allows the scale of collapse of the incinerated materials to be classified. Therefore, the combustion state of the incinerated materials in the combustion chamber R can be grasped more accurately.

(5) A collapse detection system 1 according to a fifth aspect is the collapse detection system 1 according to (4), in which the second determination unit 72 may perform the second determination using a trained model (trained model 91 for second determination) that has been trained to output a determination result regarding the possibility that the collapse has occurred when an image acquired by the acquisition unit 40 is input.

This allows for more accurate classification of the scale of collapse of the incinerated material.

(6) The collapse detection system 1 according to the sixth aspect is the collapse detection system 1 according to (4) or (5), in which the acquisition unit 40 acquires the infrared image, which is the image, at the first predetermined period and acquires a visible light image of the inside of the combustion chamber R at a second predetermined period, the second determination unit 72 makes the second determination based on one or more infrared images acquired by the acquisition unit 40, the determination unit 70 further includes a third determination unit 73 that makes a third determination regarding the collapse based on one or more visible light images acquired by the acquisition unit 40, and the first collapse determination unit 75 may determine the presence or absence of a collapse of the second scale based on the result of the first determination, the result of the second determination, and the result of the third determination.

(7) The collapse detection system 1 according to the seventh aspect is any one of the collapse detection systems 1 of (4) to (6), in which the determination unit 70 includes a disturbance determination unit 74 that determines the presence or absence of a disturbance based on a feature related to the brightness of the image acquired by the acquisition unit 40, and the determination by the first collapse determination unit 75 may be made when the disturbance determination unit 74 determines that there is no disturbance.

This makes it possible to prevent, for example, the collapse of materials to be incinerated being detected even when they have not collapsed.

(8) The collapse detection system 1 according to the eighth aspect is any one of the collapse detection systems 1 of (4) to (7), and the determination unit 70 may further include a second collapse determination unit 76 that determines the presence or absence of the first scale collapse based on a change in brightness of the multiple images acquired by the acquisition unit 40 when the first collapse determination unit 75 determines that there is no collapse of the second scale.

This allows for more accurate classification of the scale of collapse of incinerated materials.

(9) A collapse detection system 1 according to a ninth aspect is the collapse detection system 1 of (8), in which the second collapse determination unit 76 calculates a similarity between an image acquired by the acquisition unit 40 and one or more images previously acquired by the acquisition unit 40, and may determine that no collapse has occurred if the similarity does not satisfy a predetermined condition.

This makes it possible to further reduce false positives, such as detecting collapse when the material to be incinerated has not collapsed.

(10) A collapse detection system 1 according to a tenth aspect is the collapse detection system 1 of (1), in which the determination unit 70 may determine the presence or absence of a collapse by performing processing that includes inputting features extracted based on one or more images acquired by the acquisition unit 40 into a trained model 97 having a recursive structure.

This helps prevent false positives from occurring.

(11) The collapse detection system 1 according to the eleventh aspect is the collapse detection system 1 according to (10), and the determination unit 70 may include a fourth determination unit 77 that performs a fourth determination by inputting features extracted based on one or more infrared images acquired by the acquisition unit 40 into a deep learning model, a fifth determination unit 78 that performs a fifth determination by inputting features extracted based on the visible image acquired by the acquisition unit 40 into a deep learning model, and a third collapse determination unit 79 that performs a determination regarding the collapse based on features calculated based on the representative value of luminance calculated by the first calculation unit 50 and the representative value of luminance calculated by the second calculation unit 55, the result of the fourth determination, and the result of the fifth determination.

(12) The collapse detection system 1 according to the twelfth aspect is the collapse detection system 1 according to (10) or (11), in which the determination unit 70 packages features of a plurality of images acquired continuously at a predetermined time interval by the acquisition unit 40, and may input the packaged features of the plurality of images to the trained model 97 having the recursive structure.

(13) The collapse detection system 1 according to the thirteenth aspect is the collapse detection system 1 according to (12), in which the dimensions of multiple images acquired continuously at a predetermined time interval by the acquisition unit 40 are reduced using an autoencoder 96, and the features of the multiple images with reduced dimensions may be packaged.

(14) The collapse detection method according to the fourteenth aspect includes one or more computers 1100 acquiring images of the incinerated materials that have accumulated in the feeder 104 of the incineration equipment 100 and are being pushed toward the combustion chamber R at a first predetermined period, calculating a representative value of brightness based on the acquired first image, calculating a representative value of brightness based on two or more images that were acquired within the time it takes for one collapse of the incinerated materials among a plurality of images in a time series acquired before the first image, and making a judgment regarding the collapse based on the representative value of brightness based on the first image and the representative value of brightness based on the two or more images.

(15) The collapse detection system 1 according to the fifteenth aspect includes an acquisition unit 40 that acquires images of the incinerated materials that have accumulated in the feeder 104 of the incineration equipment 100 and are being pushed toward the combustion chamber R at a first predetermined period, and a collapse detection unit 41 that makes a judgment regarding the collapse based on a first input element that is a first image acquired by the acquisition unit 40, and a second input element that is two or more images captured within the time it takes for one collapse of the incinerated materials among a plurality of images in a time series acquired by the acquisition unit 40 before the first image.

The present disclosure relates to a system for detecting the collapse of materials to be incinerated within the combustion chamber of an incinerator. The collapse detection system disclosed herein can improve the accuracy of detecting the collapse of materials to be incinerated.

LIST OF SYMBOLS 1... Collapse detection system 2... Imaging device 4... Information processing device 5... Infrared camera 6... Visible light camera 8... Filter device 40... Acquisition unit 41... Collapse detection unit 42... First target area 43... Second target area 44... Third target area 45, 45a, 45b, 45c... Fourth target area 46... Fifth target area 46'... Binarized data 42a... First area 42b... Second area 42c... Third area 42d... Fourth area 42e... Fifth area 42f... Sixth area 42g... Seventh area 42h... Eighth area 42i... Ninth area 43c... Central area 43l... Left area 43r... Right area 50... First calculation unit 55... Second calculation unit 60... Third calculation unit 65... Fourth calculation unit 66... Fifth calculation unit 67... Sixth calculation unit 70...Determination unit 71...First determination unit 72...Second determination unit 73...Third determination unit 74...Disturbance determination unit 75...First collapse determination unit 76...Second collapse determination unit 77...Fourth determination unit 78...Fifth determination unit 79...Third collapse determination unit 80...Control unit 81...Fourth collapse determination unit 90...Memory unit 91...Trained model for second determination 92...Trained model for disturbance determination 93...Trained model for collapse detection 94...Trained model for fourth determination 95...Trained model for fifth determination 96...Autoencoder 97...Trained model for collapse determination 100...Incineration equipment 102...Hopper 104...Feeder 108...Furnace body 110...Extrusion device 112...Air supply device 114...Heat recovery boiler 116...Deheating tower 118...Dust collection device 120: Chimney 121: Downstream end 122: Receiving port 124: Push-out arm 126: Fire grate 128: Drying area 130: Combustion area 131: Flame 132: Post-combustion area 135: Ash 136: Air supply pipe 138: Blower 140: First flow control valve 142: Second flow control valve 143: Exhaust gas 144: Flue 145: End of furnace 146: Ash chute 1100: Computer 1110: Processor 1120: Main memory 1130: Storage 1140: Interface Fg: Garbage (material to be incinerated)
Fr: front face R: combustion chamber S: device to be controlled W1: conveying direction

Claims

An acquisition unit that acquires images of the materials to be incinerated that are piled up in a feeder of the incineration equipment and pushed toward a combustion chamber at a first predetermined period;
a first calculation unit that calculates a representative value of luminance based on a first image acquired by the acquisition unit;
A second calculation unit calculates a representative value of brightness based on two or more images captured within a time period required for one collapse of the incineration target from among a plurality of images in a time series acquired by the acquisition unit prior to the first image;
A collapse detection system comprising: a judgment unit that makes a judgment regarding the collapse based on the representative value of brightness calculated by the first calculation unit and the representative value of brightness calculated by the second calculation unit.
The collapse detection system of claim 1, wherein the second calculation unit calculates an average or median brightness value for the two or more images as a representative brightness value based on the two or more images.
The two or more images are images captured at a first time interval from each other,
The collapse detection system according to claim 1 or claim 2, wherein the two or more images are images taken before the first image for a second time period that is at least twice the first time period.
The determination unit is
a first determination unit that performs a first determination regarding the collapse based on the representative value of the luminance calculated by the first calculation unit and the representative value of the luminance calculated by the second calculation unit;
a second determination unit that performs a second determination regarding the collapse based on one or more images acquired by the acquisition unit;
A collapse detection system as described in claim 1 or claim 2, comprising a first collapse judgment unit that judges whether or not there is a collapse of a second scale, which is larger than the collapse of the first scale, based on the result of the first judgment and the result of the second judgment.
The collapse detection system according to claim 4, wherein the second determination unit performs the second determination using a trained model that has been trained to output a determination result regarding the possibility of the collapse occurring when the image acquired by the acquisition unit is input.
The acquisition unit acquires an infrared image, which is the image, at the first predetermined period, and acquires a visible light image capturing an inside of the combustion chamber at a second predetermined period,
The second determination unit performs the second determination based on one or more infrared images acquired by the acquisition unit,
The determination unit further includes a third determination unit that makes a third determination regarding the collapse based on the one or more visible light images acquired by the acquisition unit,
The collapse detection system of claim 4 , wherein the first collapse judgment unit judges whether or not a collapse of the second scale has occurred based on the result of the first judgment, the result of the second judgment, and the result of the third judgment.
the determination unit includes a disturbance determination unit that determines the presence or absence of a disturbance based on a feature amount related to luminance of the image acquired by the acquisition unit;
The collapse detection system according to claim 4 , wherein the determination by the first collapse determiner is made when the disturbance determiner determines that there is no disturbance.
The collapse detection system according to claim 4, wherein the determination unit further includes a second collapse determination unit that, when the first collapse determination unit determines that there is no collapse of the second scale, determines whether or not there is a collapse of the first scale based on a change in brightness of the multiple images acquired by the acquisition unit.
The collapse detection system of claim 8, wherein the second collapse determination unit calculates a similarity between the image acquired by the acquisition unit and one or more images previously acquired by the acquisition unit, and determines that no collapse has occurred if the similarity does not satisfy a predetermined condition.
The collapse detection system of claim 1, wherein the determination unit determines the presence or absence of a collapse by performing processing that includes inputting feature amounts extracted based on one or more images acquired by the acquisition unit into a trained model having a recursive structure.
The determination unit is
a fourth determination unit that performs a fourth determination by inputting feature amounts extracted based on one or more infrared images acquired by the acquisition unit into a deep learning model;
a fifth determination unit that performs a fifth determination by inputting the feature amount extracted based on the visible image acquired by the acquisition unit into a deep learning model;
The collapse detection system described in claim 10, further comprising: a feature calculated based on a representative brightness value calculated by the first calculation unit and a representative brightness value calculated by the second calculation unit; and a third collapse judgment unit that makes a judgment regarding the collapse based on a result of the fourth judgment and a result of the fifth judgment.
The collapse detection system according to claim 10 or 11, wherein the determination unit packages the feature amounts of a plurality of images acquired continuously at a predetermined time interval by the acquisition unit, and inputs the packaged feature amounts of the plurality of images into the trained model having the recursive structure.
The collapse detection system according to claim 12, further comprising: an autoencoder that reduces the dimensions of multiple images acquired continuously at a predetermined time interval by the acquisition unit; and packaging the features of the multiple images with reduced dimensions.
One or more computers
Acquiring images of materials to be incinerated that are accumulated in a feeder of an incineration facility and pushed toward a combustion chamber at a first predetermined period;
Calculating a representative value of luminance based on the acquired first image;
Calculating a representative value of brightness based on two or more images captured within a time period required for one collapse of the incineration target among a plurality of images in a time series acquired before the first image;
making a judgment regarding the collapse based on a representative value of brightness based on the first image and a representative value of brightness based on the two or more images.
An acquisition unit that acquires images of the materials to be incinerated that are piled up in a feeder of the incineration equipment and pushed toward a combustion chamber at a first predetermined period;
A collapse detection system comprising: a collapse detection unit that makes a determination regarding the collapse based on a first input element, which is a first image acquired by the acquisition unit; and a second input element, which is two or more images captured within the time it takes for one collapse of the material to occur among a plurality of images in a time series acquired by the acquisition unit prior to the first image.