WO2023171398A1

WO2023171398A1 - Image inspecting device, machine learning device, image inspecting method, and image inspecting program

Info

Publication number: WO2023171398A1
Application number: PCT/JP2023/006565
Authority: WO
Inventors: 基文福井
Original assignee: 住友重機械工業株式会社
Priority date: 2022-03-07
Filing date: 2023-02-22
Publication date: 2023-09-14
Also published as: TW202336704A

Abstract

An image inspecting device 300 comprises: an inspection image acquiring unit 310 for acquiring an inspection image obtained by imaging an inspection target object; a damage identifying unit 320 for identifying damage of the inspection target object in the inspection image; and a damage degree output unit 340 for outputting a degree of damage relating to the damage of the inspection target object, on the basis of a damage degree model 351 for outputting a degree of damage on the basis of an input image of damage. The machine learning device is provided with a machine learning unit 350 which employs machine learning to generate the damage degree model 351 for outputting the degree of damage relating to the damage included in the input inspection image, the machine learning being based on training data including sets comprising damage of the inspection target object, the damage being identified in the inspection image obtained by imaging the inspection target object, and the degree of damage assigned to the damage.

Description

Image inspection equipment, machine learning equipment, image inspection methods, image inspection programs

The present invention relates to an image inspection device for an object to be inspected.

When inspecting an object to be inspected, a technique is known that uses an inspection image taken of the object to be inspected. For example, Patent Document 1 discloses a technique for inspecting a civil engineering structure for damage based on an inspection image thereof.

Japanese Patent Application Publication No. 2016-21725

Conventional image inspection equipment can automatically detect damage such as cracks on the wall of the object to be inspected, but it is up to a small number of experienced workers to determine whether each detected damage requires repair or maintenance. It was done by etc.

The present invention has been made in view of these circumstances, and it is an object of the present invention to provide an image inspection apparatus and the like that can efficiently diagnose damage to an object to be inspected.

In order to solve the above problems, an image inspection apparatus according to an aspect of the present invention includes an inspection image acquisition unit that acquires an inspection image of an inspection target object, and a damage identification unit that identifies damage to the inspection target object in the inspection image. and a damage degree output unit that outputs the degree of damage to the inspection object based on a damage degree model that outputs the degree of damage based on the input damage image.

In this aspect, the damage degree model into which the inspection image of the object to be inspected is input outputs the degree of damage to the object. Can be diagnosed.

Another aspect of the present invention is a machine learning device. This device uses machine learning using training data that includes a set of damage to the inspection object identified in the inspection image taken of the inspection object and the degree of damage assigned to the damage. The system includes a machine learning unit that generates a damage degree model that outputs the degree of damage for the damage that occurs.

Yet another aspect of the present invention is an image inspection method. This method consists of an inspection image acquisition step of acquiring an inspection image of the inspection object, a damage identification step of identifying damage to the inspection object in the inspection image, and outputting the degree of damage based on the input damage image. and a damage degree output step of outputting a damage degree regarding damage to the inspection object based on the damage degree model.

It should be noted that the present invention also encompasses any combination of the above components and the conversion of these expressions into methods, devices, systems, recording media, computer programs, etc.

According to the present invention, damage to an object to be inspected can be diagnosed efficiently.

The overall configuration of a power generation facility equipped with a CFB boiler is shown. FIG. 2 is a partially cutaway perspective view showing a water pipe wall in a furnace of a boiler. FIG. 2 is a side view schematically showing an in-coke oven observation device. FIG. 2 is a diagram schematically showing the inside of a coke oven. FIG. 2 is a functional block diagram of an image inspection device. 3 schematically shows test image data stored in the test image storage unit. Damage-related information stored in a damage-related information storage unit is schematically shown. Fig. 3 schematically shows changes in the degree of damage over time. An example of a screen of a display device is shown.

Hereinafter, modes for carrying out the present invention (hereinafter also referred to as embodiments) will be described in detail with reference to the drawings. In the description and/or drawings, the same or equivalent components, members, processes, etc. are denoted by the same reference numerals, and redundant description will be omitted. The scales and shapes of the parts shown in the drawings are set for convenience to simplify the explanation, and should not be interpreted in a limited manner unless otherwise stated. The embodiments are illustrative and do not limit the scope of the present invention. Not all features or combinations thereof described in the embodiments are necessarily essential to the present invention.

The image inspection apparatus and the like of the present invention can be used to inspect any object to be inspected. Therefore, although the object to be inspected is not particularly limited, in this embodiment, an example in which the water tube wall of a boiler furnace or the furnace wall of a coke oven is the object to be inspected will be mainly described. Other examples of the inspection target will be described later.

Figure 1 shows the overall configuration of a power generation facility equipped with a CFB (Circulating Fluidized Bed) boiler. Note that instead of the CFB boiler, any other combustion equipment such as a BFB (Bubbling Fluidized Bed) boiler or a rotary kiln may be provided in the power generation equipment.

A CFB boiler includes a combustion section 1 that supplies and burns a fuel such as fossil fuel such as coal into a furnace 11 in which a fluidized material such as silica sand flows, and a steam generator that generates steam from water using the heat generated in the combustion section 1. The generation part 2, the fluidized material circulation part 3 as a circulation part that collects the fluidized material that has come out of the furnace 11 and returns it into the furnace 11, and the water supplied to the steam generation part 2, which is generated in the steam generation part 2. A heat transfer section 4 heats the steam generated by the combustion section 1 using high-temperature exhaust gas, an exhaust treatment device 5 separates and collects soot and dust in the exhaust from the heat transfer section 4, and the exhaust treatment device 5 cleans the steam. A chimney 6 is provided for discharging the oxidized exhaust gas into the atmosphere.

The combustion section 1 includes a furnace 11 as a combustion chamber. The furnace 11 has a vertically elongated cylindrical shape, and has a tapered bottom in order to increase the density of solid fuel such as coal or fluid material and enable efficient combustion. The area indicated by "A" at the bottom of the furnace 11 indicates a fluidized bed (also called a fluidized bed or sand bed) formed by a high-density fluidized material. In the fluidized bed A, powdered, particulate, or lumpy fluidized materials such as silica sand are fluidized by fluidized fluid supplied from the bottom of the furnace 11 . The solid fuel such as coal put into the fluidized bed A is efficiently combusted by repeatedly contacting the high-temperature fluidized material while being stirred within the fluidized bed A.

A perforated plate (also called a dispersion plate) 121 is provided at the bottom of the furnace 11 as a fluid permeable portion made of a porous material that allows gas to permeate therethrough. The wind box 122, which is a space directly under the perforated plate 121, supplies flowing fluid supplied from the first blower 71 as an air blower via the first flow rate control valve 71A into the furnace 11 via the perforated plate 121. It constitutes a flowing fluid supply section. The gas supplied to the bottom of the furnace 11 by the wind box 122 is used to flow the fluidized material to form the fluidized bed A, and to burn the fuel in the fluidized bed A or the freeboard B.

A second blower 72 provided in addition to the first blower 71 is an exhaust treatment device for promoting fuel combustion in the freeboard B and suppressing the generation of harmful substances such as dioxins and carbon monoxide due to incomplete combustion. 5 is supplied into the freeboard B via the second flow control valve 72A. In this way, the first blower 71 and the second blower 72 circulate at least a portion of the exhaust gas containing carbon dioxide generated by combustion in the furnace 11 from the exhaust treatment device 5 to the furnace 11 .

In order to circulate the fluidized material in the fluidized bed A, an external circulation mechanism 13 having a circulation path outside the furnace 11 is provided. The external circulation mechanism 13 includes an extraction pipe 131 that communicates with the bottom of the furnace 11 and can extract a part of the fluid material in the fluidized bed A, and controls opening and closing of the extraction pipe 131 to control the flow rate of the fluid material, that is, the extraction pipe. 131, a fluid material conveyor 133 such as a bucket conveyor that conveys the fluid material extracted by the extraction pipe 131 upward, and a fluid material conveyor 133 corresponding to the upper part of the fluidized bed A. A fluid material silo 134 provided on the outer periphery of the furnace 11 receives the fluid material conveyed by the fluid material conveyor 133, and a fluid material re-injection section 135 that reinjects the fluid material stored in the fluid material silo 134 into the furnace 11. Be prepared.

The furnace wall, which is a side wall of the furnace 11, includes a material supply section 14 that supplies fuel and other materials into the furnace 11, and a fluid material supply section 15 that supplies a fluid material for forming the fluidized bed A into the furnace 11. A starting unit 16 for starting the CFB boiler is provided. The material supply section 14 includes a funnel-shaped hopper 141 that stores materials, a crushing section 142 that crushes the material discharged from the bottom of the hopper 141 into particles, and supplies the material crushed by the crushing section 142 into the furnace 11. A feeder 143 is provided.

The material supply unit 14 supplies carbon-containing fuel containing carbon into the furnace 11 . Carbon-containing fuels are not particularly limited, and include, for example, various types of coal such as anthracite, bituminous coal, and brown coal, biomass fuel, sludge, and waste wood. These carbon-containing fuels generate carbon dioxide when burned in the furnace 11. However, biomass fuel is a carbon-neutral fuel with little or no net carbon dioxide emissions. The crushing unit 142 in the material supply unit 14 crushes the material into particles before being supplied to the furnace 11 . The required amount of the granular material crushed by the crushing section 142 is fed into the furnace 11 by a feeder 143 whose rotation speed can be controlled.

The fluid supply unit 15 that supplies the fluid for forming the fluidized bed A includes a funnel-shaped fluid hopper 151 that stores the fluid and the fluid that is discharged from the bottom of the fluid hopper 151 into the furnace 11. A fluid material feeder 152 is provided. By controlling the rotation speed of the fluid material feeder 152, a required amount of fluid material is fed into the furnace 11.

The starting section 16 that starts the CFB boiler includes a starting fuel storage section 161, a starting fuel control valve 162, and a starting burner 163. The startup fuel storage section 161 stores heavy oil as carbon-containing fuel. The starting fuel control valve 162 controls the amount of heavy oil supplied from the starting fuel storage section 161 to the starting burner 163. Specifically, the startup fuel control valve 162 is opened when the CFB boiler is started, and the heavy oil stored in the startup fuel storage section 161 is supplied to the startup burner 163. The starting burner 163 heats the fluidized material in the fluidized bed A with flame generated by combustion of heavy oil supplied from the starting fuel control valve 162 . Since the starting burner 163 is provided to be inclined downward, the surface of the fluidized bed A formed by the fluidized material is directly heated, and the temperature of the fluidized bed A and the inside of the furnace 11 is efficiently raised. In this way, the starting burner 163 heats the sand-like fluidized bed A from above, so it is also called an over-sand burner.

After starting the CFB boiler in which the temperature inside the fluidized bed A and the furnace 11 has risen sufficiently, specifically, after it becomes possible to burn the fuel or material supplied from the material supply section 14 in the fluidized bed A, starting fuel control is performed. The valve 162 is closed and the supply of heavy oil to the starting burner 163 is stopped. In the subsequent normal operating state, fuel supplied from the material supply section 14 is burned in the high-temperature furnace 11.

The combustion section 1 of the CFB boiler has been described in detail above. Next, the configuration of the CFB boiler other than the combustion section 1 will be explained. The steam generation unit 2 includes a drum 21 that stores water for generating steam, a water supply pipe 22 that supplies water to the drum 21, and a water pipe 23 that guides water in the drum 21 into the high-temperature furnace 11 and heats it. A steam pipe 24 is provided for discharging steam generated from water heated in the water pipe 23 from the drum 21 as the output of the CFB boiler. The steam output from the steam pipe 24 rotates the steam turbine of the generator 25, so that the power generation equipment generates electricity. The water supply pipe 22 constitutes an economizer that preheats the water supply by meandering through the heat transfer section 4 through which the high temperature exhaust gas from the combustion section 1 passes, and the steam pipe 24 constitutes a heat transfer section through which the high temperature exhaust gas from the combustion section 1 passes. 4 constitutes a superheater that superheats steam.

The fluid material circulation unit 3 includes a cyclone 31 that separates and collects granular fluid material from the exhaust gas discharged from the upper part of the furnace 11, and a seal pot 32 that returns the fluid material collected by the cyclone 31 into the furnace 11. Equipped with The cyclone 31 is a cyclone-type powder separator having a substantially cylindrical upper part and a substantially conical lower part, and generates an airflow that descends spirally along the inner wall. The granular fluidized material contained in the exhaust gas from the furnace 11 comes into contact with the inner wall of the cyclone 31 when descending spirally along the airflow and is collected.

A seal pot 32 provided below the cyclone 31 is filled with a fluid material to prevent unburned gas, etc. from flowing back from the furnace 11 to the cyclone 31. The granular fluidized material filled in the seal pot 32 is gradually returned to the furnace 11 as it is pushed out by the weight of the fluidized material newly collected by the cyclone 31. The exhaust treatment device 5 separates and collects soot and dust in the exhaust from the heat transfer section 4.

FIG. 2 is a partially cutaway perspective view showing a water pipe wall 80 that constitutes the inner wall of a furnace 11 such as a CFB boiler or a BFB boiler, which is an example of an object to be inspected. The water tube wall 80 of the furnace 11 is composed of a plurality of pipes 82 extending in the vertical direction and fins 84 connecting each adjacent pipe 82. Water, other liquids, and their vapors pass through each pipe 82 . Since the water tube wall 80 faces the high-temperature furnace 11, it may be damaged by heat. Furthermore, there is a possibility that fuel such as coal or ash burned in the furnace 11 collides with or adheres to the water pipe wall 80, leading to damage. Furthermore, there is a possibility that water or steam inside the pipe 82 that becomes high-pressure in a high-temperature environment may damage the water pipe wall 80 from within the pipe 82 . As described later, according to the image inspection apparatus 300 according to the present embodiment, such damage in the water pipe wall 80 can be efficiently diagnosed.

A camera 30 (not shown in FIG. 2) as a photographing device for photographing an inspection image of the water tube wall 80 of the furnace 11 is a moving body that moves the camera 30, for example, a slide mechanism such as the extrusion device 200 shown in FIG. , attached to a drone, robot, elevator, etc. that can move along the water pipe wall 80. The camera 30 continuously photographs the furnace wall 91 while moving along the water pipe wall 80 inside the furnace 11 together with the moving body. The camera 30 may be a still camera that continuously takes still images, or a video camera that takes moving images.

FIG. 3 is a side view schematically showing an in-furnace observation device in which the image inspection device of this embodiment is used. This furnace interior observation device is used to observe the inside of a coke oven carbonization chamber (hereinafter also simply referred to as a coke oven). FIG. 4 is a diagram schematically showing the inside of the coke oven.

The coke oven 90 is a narrow oven with a pair of brick oven walls 91 facing each other. Each oven wall 91 extends from an oven inlet 92 on one side of the coke oven 90 to an oven outlet 93 on the other side, and has a total length of, for example, more than ten meters. The distance between the opposing furnace walls 91 is, for example, several tens of centimeters. The height from the bottom 95 to the ceiling 94 of the coke oven 90 is, for example, several meters.

The extrusion device 200 shown in FIG. 3 repeatedly moves back and forth within the coke oven 90. On the outward journey, the extrusion device 200 is inserted into the coke oven 90 from the oven inlet 92 and pushes out the coke C generated by carbonization in the coke oven 90 to the oven outlet 93. On the return trip, the extrusion device 200 returns inside the coke oven 90 from the oven outlet 93 to the oven inlet 92. The extrusion device 200 includes a push plate 210 and a beam 220, and the beam 220 connects the push plate 210 to a drive device (not shown). This driving device allows the push plate 210 to move between the furnace inlet 92 and the furnace outlet 93 of the coke oven 90. Since the push plate 210 has substantially the same cross-sectional shape as the coke oven 90, the movement of the push plate 210 pushes the coke C toward the oven outlet 93.

A camera 30 as a photographing device for photographing an inspection image is attached to an extrusion device 200 as a movable body for moving the camera 30, and moves together with the extrusion device 200 within the coke oven 90 between the furnace inlet 92 and the furnace outlet 93. The furnace wall 91 is continuously photographed. The camera 30 may be a still camera that continuously takes still images, or a video camera that takes moving images. The camera 30 is attached to the back surface of the push plate 210 (the right side in FIG. 3) or to a support stand (not shown) installed behind the push plate 210.

The camera 30 may include two cameras attached to the left and right furnace walls 91, respectively, and captures front-view images of the left and right furnace walls 91. In order to photograph the entire left and right furnace walls 91, the two cameras may photograph while changing the angle in the vertical direction. Note that the camera 30 may be mounted so as to face in the opposite direction (to the right in FIG. 3) to the direction in which the coke C is extruded by the extrusion device 200 so that the view is not obstructed by the push plate 210 or the coke C. The camera 30 in this case is installed directly facing the furnace inlet 92 and can take perspective images of the furnace walls 91 on both the left and right sides. As a heat measure to protect the camera 30 from the high-temperature environment (eg, 1000° C. or higher) inside the coke oven 90, the camera 30 may be housed in a heat-resistant housing or a cooling box, for example.

FIG. 5 is a functional block diagram of the image inspection apparatus 300 according to this embodiment. The image inspection apparatus 300 includes an inspection image acquisition section 310, a damage identification section 320, a damage position acquisition section 331, a damage related information acquisition section 332, a damage degree output section 340, a machine learning section 350, and a damage degree prediction section. 360 and a maintenance necessity determining section 370. These functional blocks are realized through the collaboration of hardware resources such as the computer's central processing unit, memory, input devices, output devices, and peripheral devices connected to the computer, and the software that is executed using them. . Regardless of the type of computer or installation location, each of the above functional blocks may be realized using the hardware resources of a single computer, or may be realized by combining hardware resources distributed across multiple computers. .

The camera 30 and the inspection image storage unit 301 include an image input unit that inputs an inspection image of the water tube wall 80 of the furnace 11 and the furnace wall 91 of the coke oven, which are the inspected surfaces of the inspection object, into the image inspection apparatus 300. Configure. The display device 40 displays the processing contents of the image inspection device 300 and the like. The operation unit 50 is configured with an input device such as a touch panel integrated with the display device 40 or a keyboard and a mouse separate from the display device 40, and generates various control information for the image inspection apparatus 300 in response to user operations. The computer may be programmed to autonomously perform some or all of the operations performed by the operation unit 50.

The inspection image storage unit 301 stores a group of inspection images of the water pipe wall 80 and the furnace wall 91 taken by the camera 30. The inspection image storage section 301 may be the built-in memory of the camera 30, or may be a general-purpose removable medium such as a memory card. Alternatively, storage outside the boiler or outside the coke oven 90 that can communicate with the camera 30 by wire or wirelessly may be used. The image inspection apparatus 300 performs various processes described below on the inspection image group stored in the inspection image storage unit 301. Here, if the camera 30 and the image inspection device 300 can communicate by wire or wirelessly, and if the image inspection device 300 can acquire and process the inspection images taken by the camera 30 in real time, all the inspection images may not be There is no need to store it in the storage unit 301, and the inspection image storage unit 301 can be unnecessary or have a small capacity.

FIG. 6 schematically shows the test image data stored in the test image storage unit 301. Inspection image data 42 photographed by the camera 30 is stored in the inspection image storage unit 301 together with metadata such as damage degree data 43, photographing date and time, and photographing position, which will be described later. The damage level data 43 includes various types of damage 431 to 433 identified by a damage identification unit 320 (described later) in the inspection image data 42, and at least two levels of damage imparted to each damage 431 to 433 by a damage level output unit 340 (described later). Show degree. The photographing date and time is the date and time (date and/or time) when the inspection image was photographed by the camera 30, and the photographing position is the position of the inspection image photographed by the camera 30. A specific example of the photographing position will be described later regarding the damage position acquisition unit 331, but for example, it may include a position measured by a positioning sensor mounted on a camera, an altitude measured by an altimeter, an attitude or direction measured by an inertial sensor, etc. The imaged position or imaged part of the inspection object is derived directly or indirectly based on the imaged area. In the illustrated example, (100, 100, 100) are the three-dimensional coordinates of the photographed position in the inspection object. In addition to or in place of these metadata, information that can identify the photographed inspection object (for example, information such as "the right oven wall of the Y carbonization chamber in the X coke oven") is added to the inspection image. It may be stored in the inspection image storage unit 301 as metadata of the data 42. Note that in this figure and subsequent figures, an image of the oven wall 91 of the coke oven 90 is shown schematically as an example of an inspection image for convenience, but the explanation regarding these will be given to other inspection objects, especially in the furnace 11 of the boiler. It can be similarly applied to image inspection of the water tube wall 80.

The inspection image acquisition unit 310 acquires an inspection image of the object to be inspected from the camera 30 or the inspection image storage unit 301. Here, when the inspection image acquisition unit 310 acquires an inspection image from the inspection image storage unit 301, the user uses the inspection image specifying unit 51 in the operation unit 50 to specify the date and time of the inspection image to be acquired, the shooting position, the location of the image taken, etc. Various conditions (metadata as shown in FIG. 6) regarding the inspection image of the inspection object etc. can be specified. In this case, the test image acquisition unit 310 searches the test image storage unit 301 for a test image that matches the conditions specified by the test image specifying unit 51, and selects part or all of the one or more pieces of test image data 42 that are found. It is acquired from the inspection image storage unit 301 along with the metadata (FIG. 6).

The damage identification unit 320 uses a known image inspection technique to identify damage to the inspection object in the inspection image acquired by the inspection image acquisition unit 310. Damages, defects, and abnormalities occurring in the oven wall 91 of the coke oven 90, which serves as the surface to be inspected, include cracks and holes in the oven wall 91, spalling where the oven wall 91 breaks due to distortion due to rapid heating or cooling, Examples include adhesion of carbon derived from coal, which is a raw material for coke, to the furnace wall 91 and deterioration of the joints of bricks forming the furnace wall 91. In addition, damage, defects, and abnormalities occurring in the water tube wall 80 in the furnace 11 of the boiler, which serves as the inspection surface of the inspection object, include damage caused by the heat of the high-temperature furnace 11 and damage caused by combustion in the furnace 11, as described above. Examples include damage caused by collision or adhesion of fuel such as coal or ash, and damage from within the pipe 82 due to water or steam that becomes high pressure in a high temperature environment. The damage identified by the damage identification unit 320 (damages 431 to 433 in FIG. 6, etc.) is converted into metadata (damage degree The data 43) are stored in the inspection image storage unit 301.

The damage position acquisition unit 331 acquires the position of the damage on the inspection object identified by the damage identification unit 320. For example, the position and orientation of the camera 30 when an inspection image including damage to the inspection target is taken by a position sensor or an inertial sensor that measures the position and orientation (direction) of the moving body on which the camera 30 is mounted or the camera 30 itself. can be recognized. The moving body that moves the camera 30 may be a sliding mechanism such as the extrusion device 200 shown in FIG. 3, or a circulating fluidized bed (CFB) boiler, a fluidized bed (BFB) boiler, etc. Drones, robots, etc. that are used to photograph inspection target parts such as the inner or outer walls of boiler water pipes and smoke pipes, the outer and inner walls of buildings, other industrial machines, industrial structures, and high places of industrial equipment with the camera 30, It may be an elevator, etc. If the drone, robot, elevator, etc. or the camera 30 itself is equipped with a positioning sensor such as a GPS (Global Positioning System), an altimeter, an inertial sensor, etc., the measurement results can be used to identify the location of damage.

In addition, when a user (person) photographs an object to be inspected with the camera 30, the camera 30 uses a positioning sensor such as a GPS, an inertial sensor, etc. installed in a mobile device such as a smartphone used by the user. The position of damage on the photographed inspection object may be identified. Furthermore, the position or region of the inspection object photographed by the camera 30 may be identified by comparing the two-dimensional or three-dimensional inspection image photographed by the camera 30 with the known structure of the inspection object. In addition, the position data or orientation data of these inspection images, such as the absolute position measured by a positioning sensor such as GPS, the absolute altitude measured by an altimeter, the attitude measured by an inertial sensor, and the inspection target It is preferable that the relative position in the inspection object, the region on the inspection object, etc. be stored in the inspection image storage unit 301 together with the inspection image data 42 as metadata of the inspection image data 42 ("imaging position" in FIG. 6).

The damage-related information acquisition unit 332 acquires information related to damage to the inspection object specified by the damage identification unit 320 from the damage-related information storage unit 302. The related information stored in the damage-related information storage unit 302 includes the repair history of the damage to the inspection object identified by the damage identification unit 320, the location on the inspection object of the damage to the inspection object identified by the damage identification unit 320 (Fig. 6 ), the damage level output by the damage level output unit 340 in the past for damage similar to the damage to the inspection target identified by the damage identification unit 320, and the damage level that has caused past incidents and serious accidents. and/or information regarding the damage caused. For convenience, the damage-related information storage section 302 will be described below as a separate storage section from the inspection image storage section 301, but since these stores mutually overlapping information or data, it is best to configure them as an integrated storage section. preferable.

FIG. 7 schematically shows damage-related information stored in the damage-related information storage section 302. The damage-related information may include part or all of the inspection image data 42 and metadata "damage degree data" (43), "photography date and time", and "photography position" stored in the inspection image storage unit 301 in FIG. 6. In addition to or in place of these data, the damage-related information includes a repair history 44 of the damages 431 to 433 identified by the damage degree data 43 and an incident history 45 of the damages 431 to 433 identified by the damage degree data 43. But that's fine.

The repair history 44 includes past repair reports and maintenance reports (hereinafter collectively referred to as repair reports) for the damage 431 to 433 specified by the damage degree data 43 and/or the photographed site of the inspection image data 42. The repair report is an electronic document that records the details of repairs and maintenance performed in the past on the photographed site of the inspection image data 42, and the results of observations such as damage at that time, etc., for each implementation date. The repair report may be freely written in a free format, or the contents may be selected or input for each predetermined item. In the case of free description, for example, "The crack was 10 cm long and 2 mm wide in 2019, but has grown to 12 cm long and 2 mm wide in 2020. However, the urgency and severity at this point is low. Therefore, it was determined that the crack was not subject to repair, but required observation.This crack is located at the upper corner of the fireproof wall, and was repaired once in 2015,'' etc. in the repair report. recorded. In order to be able to extract necessary damage-related information from such free descriptions, it is preferable to classify and give meaning to the data in advance through Natural Language Processing (NLP) or the like.

Each item included in the repair report may be classified in advance according to predetermined criteria. For example, the positions of the imaged parts and/or damages 431 to 433 in the inspection image data 42 (in the free description example above, "the upper corner of the fireproof wall") are (ceiling, left side, right side, corner, In the position vector P defined as fireproof wall upper part,...) ^t , it is classified as P=(0,0,0,1,1,...) ^t . Here, the fourth vector element of the "corner" and the fifth vector element of the "top of the fireproof wall" are set to "1", and the remaining vector elements are set to "0", so that the above free description In the example, the position "upper corner of the fireproof wall" is expressed. Similarly, past repair times ("2015" in the free description example above) are (repaired more than 10 years ago, repaired 10-9 years ago, repaired 9-8 years ago, 8-7 years ago). In the repair time vector M defined as 8.5 years ago, M = (0, 0, 0, 1 ^{, 0,...) t} ^, It is expressed as In addition, for each damage 431 to 433 identified in the damage degree data 43, the type of damage ("cracks" in the free description example above) is defined as (cracks, holes, spalling, adhesion of foreign matter, deterioration of joints, etc.). ^. It may be classified according to a damage degree vector S defined as damage degree 2, damage degree 1) ^t .

The incident history 45 includes reports (hereinafter collectively referred to as incident reports) regarding past incidents and serious accidents of the damage 431 to 433 identified by the damage degree data 43 and/or the part photographed in the inspection image data 42. The incident report is an electronic document that records the details of incidents and serious accidents that occurred in the past in the photographed part of the inspection image data 42, by date of occurrence. The incident report may be freely written in a free format, or the contents may be selected or input for each predetermined item. In the case of free writing, for example, "On April 30, 2003, a part of the bricks collapsed from the cracks formed in the bricks and joints on the right furnace wall of the coke chamber Y in the coke oven X." The contents are recorded in the incident report. Similar to the repair report mentioned above, it is preferable to classify and give meaning to the data in advance through natural language processing etc. so that necessary damage-related information can be extracted from such free descriptions for incident reports. Each item included in the list may be classified in advance according to predetermined criteria.

For example, the location of the incident occurrence (in the free description example above, "the right furnace wall of the Y coking chamber in the X coke oven") can be classified in the same way as the position vector P, and the incident occurrence time (the In the free description example, "2003") can be classified in the same way as the repair time vector M above, and the type of damage that caused the incident (in the free description example above, "crack part") can be classified as the above damage type. It can be classified in the same way as the vector D, and the degree of damage at the time of the damage that caused the incident can be classified in the same way as the damage degree vector S described above. Further, for example, five levels of incidents may be classified by an incident level vector I defined as (incident level 5, incident level 4, incident level 3, incident level 2, incident level 1) ^t . According to such incident history 45, in what position and part of the object to be inspected, what type of damage is likely to become severe or serious, and in the worst case, what level of risk will the incident develop? I can recognize whether something is there or not. In addition, by referring to the repair history 44 and the incident history 45 together, it is possible to make suggestions such as when and what kind of repair should be performed to effectively prevent incidents and serious accidents from occurring regarding identified damage. You can also get

Among the damage-related information stored in the damage-related information storage unit 302 as described above, the damage-related information acquisition unit 332 extracts information related to damage to the inspection object specified by the damage identification unit 320. For example, the past damage level (damage level data 43) of the damage itself identified by the damage identification unit 320, past repair history 44, and past incident history 45 are extracted by the damage-related information acquisition unit 332 as damage-related information. . In addition, the damage-related information acquisition unit 332 extracts repair history 44 and incident history 45 of other damages that are similar to the damage identified by the damage identification unit 320 in terms of object to be inspected, location, type, degree of damage, etc. as damage-related information. can. In this way, by also referring to the repair history 44 and incident history 45 for other similar damages, it is possible to determine whether the damage to the inspection object identified by the damage identification unit 320 requires repair, the risk of developing into an incident or serious accident, and other information. Then, the damage degree output unit 340, which will be described next, can diagnose the "damage degree" with high accuracy.

Note that the degree of damage may be expressed as the severity, degree of progression, degree of danger, seriousness, etc. of the damage. Here, even if damage (for example, a crack) has the same size and shape, the degree of damage may vary depending on where it is formed. For example, based on the incident history 45 (Fig. 7) of various past damages, it is possible to obtain the knowledge that a specific type of damage formed in a specific part of the object to be inspected is likely to become severe and lead to a serious accident. Specifically, if it is known that cracks formed at the corners of the water pipe wall 80 or the furnace wall 91 are likely to become severe and lead to a serious accident, the damage level output The portion 340 imparts a higher degree of damage to cracks formed at corners than to cracks formed outside corners. Additionally, damage that has a long elapsed time since the last or most recent repair that can be recognized from the repair history 44 (Figure 7) is generally given a high damage rating, and damage that has a short time that has elapsed since the last or most recent repair is generally given a low damage rating. Damage level is given.

The damage level output unit 340 divides the damage of the inspection object identified by the damage identification unit 320 into at least two stages (specifically outputs the degree of damage, for example, in 5 stages from "1" to "5" or continuous numerical values). The damage level model 351 is generated by a machine learning unit 350 that constitutes a machine learning device. The machine learning unit 350 identifies damage to the inspection object identified in the inspection image taken of the inspection object, and damage that has been added (labeled) artificially or through a labeling tool or an annotation tool. A damage degree model 351 is generated by machine learning in a neural network or the like using exhaustive training data including a set of degrees.

The training data for the machine learning unit 350 to generate the damage degree model 351 is, for example, the corresponding damage degree (“1” and “2” in the example of FIG. 6, respectively) for each damage 431, 432, and 433 in FIG. , "1"). Here, the training data includes some or all of the data stored in the inspection image storage unit 301 shown in FIG. Alternatively, some or all of the data stored in the damage-related information storage unit 302 shown in FIG. It is preferable that each damage 431, 432, and 433 be labeled in the "history 45". As a result, the damage level output unit 340 or the damage level model 351 can output not only the inspection image data 42 acquired by the inspection image acquisition unit 310 and the damage data (431 to 433) specified by the damage identification unit 320, but also the inspection image storage unit 301 and/or other available data stored in the damage-related information storage unit 302, the degree of damage of each damage (431 to 433) can be diagnosed with high accuracy.

In particular, as schematically shown in FIG. 5, the damage level output unit 340 or the damage level model 351 includes the inspection image data 42 acquired by the inspection image acquisition unit 310 and the damages 431 to 433 acquired by the damage position acquisition unit 331. Damage identification is performed based on various information or data such as the location of the damage 431 to 433 (FIG. 7) acquired by the damage-related information acquisition unit 332 (“shooting position” in FIG. 6 and/or FIG. 7), The unit 320 outputs, for example, a five-level damage degree for the damage 431 to 433 on the inspection object specified. The damage degree output by the damage degree output unit 340 for each damage 431 to 433 is reflected in the damage degree data 43 in the inspection image storage unit 301 (FIG. 6) and/or the damage related information storage unit 302 (FIG. 7).

The damage level prediction unit 360 calculates the predicted date and time at the future date specified by the predicted date and time designation unit 52 in the operation unit 50 based on the damage level of the damage 431 to 433 of the inspection object outputted in the past by the damage level output unit 340. The degree of damage of the damage 431 to 433 is predicted. As schematically shown in FIG. 8, the damage degree output unit 340 outputs the damage degree "1" → "2" → "2" at the past date and time T ₁ → T ₂ → T ₃ for the same damage. It is assumed that there is In this figure, dotted circles represent five levels of damage levels output by the damage level output unit 340 at each date and time T ₁ , T ₂ , and T ₃ . However, as shown by the black circle, the damage level output unit 340 or the damage level model 351 actually calculates the damage level below the decimal point, and this information is used to predict the damage level in the damage level prediction unit 360. used.

The damage level prediction unit 360 is configured to calculate the predicted date and time specified by the predicted date and time designation unit 52 based on the change over time in the damage level (black circle) calculated by the damage level output unit 340 at past dates and times T ₁ , T ₂ , and T ₃ . The degree of damage of the damage at the predicted future date and time _TP is predicted. In this prediction, a known statistical method such as an autoregressive integrated moving average (ARIMA) model can be used. In the example of FIG. 8, the damage level prediction unit 360 predicts the damage level of the damage at the predicted date and time _TP to be "5" (the most serious of the five levels). Note that the damage degree prediction section 360 refers to changes over time in the degree of damage (damage degree data 43) for other similar injuries recorded in the inspection image storage section 301 and/or the damage related information storage section 302. Then, the future transition of the degree of damage to be predicted may be predicted.

The maintenance necessity determination unit 370 determines whether or not maintenance is required for the inspection target object based on the degree of damage 431 to 433 of the inspection target object output by the damage level output unit 340. For example, if a maintenance implementation standard is set such that maintenance is to be performed when the damage level reaches "4" for the damage to be determined, if the damage level output by the damage level output section 340 is "3" or lower, If the maintenance necessity determination section 370 determines that maintenance is not necessary, and the damage degree output by the damage degree output section 340 is "4" or more, the maintenance necessity determination section 370 determines that maintenance is necessary. Further, the maintenance implementation standard may be set based on the degree and area of damage. For example, when the area or area increase rate of damage with a damage level of "2" or higher exceeds a predetermined threshold value, it may be determined that maintenance is necessary. Note that the maintenance necessity determining unit 370 may present a future date on which maintenance should be performed. In the example of FIG. 8, the maintenance necessity determining unit 370 presents the date and time when the damage level of the damage to be determined will reach “4” by the damage level predicting unit 360 as the recommended maintenance date and time _TM .

FIG. 9 shows an example screen of the display device 40. In the upper left corner, inspection image data 42 (FIG. 6) as an inspection object is displayed. The accompanying information display area 41 below displays various information accompanying the examination image data 42 stored in the examination image storage section 301 and/or the damage-related information storage section 302, specifically, the date and time of photographing, The location, repair history 44, incident history 45, etc. are displayed. Although not shown in FIG. 5, since the inspection image storage section 301 and/or the damage-related information storage section 302 are connected to the display device 40, the inspection image storage section 301 and/or the damage-related information storage section 302 are Any information stored in the related information storage unit 302 can be displayed. In the past data display area 46, inspection image data and damage degree data (43) obtained when a portion of the inspection object that is the same as or overlapping with the inspection image data 42 was photographed in the past are displayed together with the date and time of photographing. Based on changes in damage level data over time, it is possible to understand at a glance the degree of severity and progression of each injury (i.e., changes in damage level over time). In this way, the image inspection apparatus 300 or the damage degree output unit 340 outputs the damage degree output in the past at the position of damage on the inspection object (in FIG. 9, the damage degree data in the past data display area 46 (omitted).

In the usage information selection area 53, the operation unit 50 can select information to be considered when the damage level output unit 340 or the damage level model 351 outputs the level of damage included in the inspection image data 42. In the illustrated example, check boxes are provided for two pieces of information: "location" and "repair time." If the "Position" checkbox is checked by the operation unit 50, the damage level output unit 340 or the damage level model 351 outputs the damage level of the damage included in the inspection image data 42. For example, consider the "imaging position" in FIG. 6 and/or FIG. 7. For example, the degree of damage of the damage in the inspection image data 42 is output in consideration of the incident history 45 of other damages whose positions are similar to the damage in the inspection image data 42 identified by the damage identification unit 320. In addition, if the “Repair time” checkbox is checked by the operation unit 50, the damage level output unit 340 or the damage level model 351 outputs the damage level of the damage included in the inspection image data 42. The last or latest repair time of the damage included in the repair history 44 (FIG. 7) is considered. That is, the degree of damage is output taking into consideration not only the external characteristics of the damage that can be ascertained from the inspection image data 42, but also the contents of past repairs or maintenance for the damage.

When the execution button 55 is pressed using the operation unit 50 with necessary information selected in the usage information selection area 53, the damage level output unit 340 or the damage level model 351 outputs the damage level data 43 (FIGS. 6 and 6) for the inspection image data 42. /or data similar to that in FIG. 7) is generated and displayed. Thereby, each damage included in the inspection image data 42 of the inspection target and the degree of damage can be grasped at a glance. Note that the damage degree data 43 may display a difference from the most recent (in the example of FIG. 9, the latest data in January 2021 in the past data display area 46) damage degree data.

In the prediction information designation area 54, necessary information for the damage degree prediction section 360 to predict the degree of damage included in the inspection image data 42 at a future date can be input using the operation section 50. In the illustrated example, "ARIMA" (autoregressive integrated moving average) is entered in the "Prediction Model" column as the statistical method used for prediction, and the future prediction date and time T _P (Figure 8) is entered. “2023/1” is input in the “forecast time” column by the prediction date and time specifying unit 52. When the execution button 55 is pressed on the operation unit 50 with the necessary information entered in the prediction information specification area 54, the damage degree prediction unit 360 selects a future date (in the example of FIG. 9, January 2023) for the inspection image data 42. ) is predicted and displayed. As a result, for each damage included in the inspection image data 42 of the inspection target, the damage degree (43) at the current or shooting date and time (January 15, 2022 in the example of FIG. 9) and the damage degree (47) at a future date are determined. ) can be understood at a glance by comparing them. Note that a plurality of prediction dates and times may be input in the "prediction time" column, and the damage degree prediction unit 360 may predict and display a plurality of damage degree data 47 at the plurality of prediction dates and times. .

In addition to or in place of the damage degree data 47 predicted by the damage degree prediction unit 360, the display device displays changes in the degree of damage over time from the past to the future as shown in FIG. 8, and recommended maintenance dates and times _TM . 40 may be displayed. When the image save button 56 provided at the bottom of the screen in FIG. The information updated on the screen is stored in the inspection image storage section 301 (FIG. 6) and/or the damage-related information storage section 302 (FIG. 7). Further, when the repair flag 57 is pressed using the operation unit 50, a finding that early repair is necessary is recorded for the part of the inspection object photographed using the inspection image data 42 ("photographing position" in FIG. 9). In addition, the organization and personnel in charge of repairs are notified.

The present invention has been described above based on the embodiments. Those skilled in the art will understand that the embodiments are merely illustrative, and that various modifications can be made to the combinations of the constituent elements and processing processes, and that such modifications are also within the scope of the present invention.

In the embodiment described above, the boiler and the coke oven 90 were exemplified as the objects to be inspected by the image inspection apparatus 300, but the objects to be inspected are not limited thereto. For example, the inspection target may be various industrial machines such as construction machinery (including boilers and coke ovens), social infrastructure such as bridges, various industrial structures such as environmental plants and water treatment facilities, or other industrial equipment. The surface to be inspected may be an internal or external surface of such industrial equipment. Further, a camera that takes a group of images of the surface to be inspected can be attached to a moving body of any configuration that can move along the surface to be inspected. For example, the camera may be attached to a flying object such as a so-called drone.

Note that the functional configuration of each device described in the embodiments can be realized by hardware resources or software resources, or by cooperation between hardware resources and software resources. A processor, ROM, RAM, and other LSIs can be used as hardware resources. Programs such as operating systems and applications can be used as software resources.

11 Furnace, 30 Camera, 40 Display device, 42 Inspection image data, 43 Damage data, 44 Repair history, 45 Incident history, 46 Past data display area, 47 Damage data, 50 Operation unit, 80 Water pipe wall, 90 Coke Furnace, 91 Furnace wall, 200 Extrusion device, 300 Image inspection device, 301 Inspection image storage unit, 302 Damage related information storage unit, 310 Inspection image acquisition unit, 320 Damage identification unit, 331 Damage position acquisition unit, 332 Damage related information acquisition 340 damage level output unit, 350 machine learning unit, 351 damage level model, 360 damage level prediction unit, 370 maintenance necessity determination unit.

Claims

an inspection image acquisition unit that acquires an inspection image of the inspection object;
a damage identification unit that identifies damage to the inspection object in the inspection image;
a damage degree output unit that outputs a degree of damage to the inspection object based on a damage degree model that outputs a degree of damage based on an input damage image;
An image inspection device comprising:
further comprising a damage position acquisition unit that acquires the position of damage on the inspection object,
The damage degree model outputs a damage degree based on the input damage image and position.
The image inspection device according to claim 1.
further comprising a damage-related information acquisition unit that acquires information related to damage to the inspection object,
The damage degree model outputs a damage degree based on the input damage image and related information.
The image inspection apparatus according to claim 1 or 2.
The information related to the damage includes the repair history of the damage, the location of the damage on the object to be inspected, the degree of damage output by the damage level output unit in the past for damage similar to the damage, and the cause of past incidents. 4. The image inspection apparatus according to claim 3, wherein the image inspection apparatus includes at least one of information regarding damaged damage.
1 . The damage level prediction unit further comprises a damage level prediction unit that predicts the damage level of the damage at an input future date based on the damage level of the damage to the inspection object outputted in the past by the damage level output unit. 10 . 5. The image inspection device according to any one of 4 to 4.
Any one of claims 1 to 5, further comprising a maintenance necessity determination unit that determines whether or not maintenance of the inspection target is required based on the degree of damage to the inspection target outputted by the damage level output unit. The image inspection device described in .
The image inspection apparatus according to any one of claims 1 to 6, wherein the damage degree output unit displays a damage degree outputted in the past at a position of damage on the inspection object.
Damage included in the input inspection image is determined by machine learning using training data that includes a set of damage to the inspection target identified in the inspection image taken of the inspection target and a damage degree assigned for the damage. A machine learning device that includes a machine learning unit that generates a damage degree model that outputs a degree of damage.
an inspection image acquisition step of acquiring an inspection image of the inspection object;
a damage identification step of identifying damage to the inspection object in the inspection image;
a damage degree output step of outputting a damage degree regarding damage to the inspection object based on a damage degree model that outputs a damage degree based on an input damage image;
An image inspection method comprising:
an inspection image acquisition step of acquiring an inspection image of the inspection object;
a damage identification step of identifying damage to the inspection object in the inspection image;
a damage degree output step of outputting a damage degree regarding damage to the inspection object based on a damage degree model that outputs a damage degree based on an input damage image;
An image inspection program that causes a computer to execute