WO2021246210A1 - Machine learning data generation device, machine learning data generation method, program, and learning data set - Google Patents

Abstract

For the purpose of learning a machine learning model in a gas leakage detection device, the present invention can efficiently generate a large amount of learning data under various conditions and contribute to learning accuracy improvement. This machine learning data generation method comprising a step for converting three-dimensional gas distribution image data indicating an existence range of a gas leaking from a gas leakage source into a three-dimensional space into two-dimensional gas distribution image data observed from a prescribed view point position and a step for generating, as training data for machine learning, the two-dimensional gas distribution image data and prescribed characteristic information about the gas.

Description

Machine learning data generator, machine learning data generation method, program and learning dataset

The present disclosure relates to a machine learning data generator, a machine learning data generation method, a program, and a learning data set, and more particularly to a machine learning data generation method used in a gas leak detection device.

Equipment that uses gas, such as production facilities that produce natural gas and petroleum, production plants that produce chemical products using gas, gas transmission equipment, petrochemical plants and thermal power plants, and steel-related facilities (hereinafter referred to as "" (Sometimes referred to as "gas equipment"), the danger of gas leakage is recognized due to aging of the facility and operational mistakes, and a gas detection device is installed to minimize gas leakage.

In this gas detection, in addition to gas detection devices that utilize the fact that the electrical characteristics of the probe change when gas molecules come into contact with the detection probe, in recent years, infrared moving images have been taken using the infrared absorption characteristics of gas. As a result, an optical gas leak detection method for detecting a gas leak in the inspection region has been adopted (for example, Patent Documents 1 and 2).

The gas detection method using infrared moving images has the advantage that the emission state such as gas flow and the leak position can be easily detected because the gas can be visualized by the image. In addition, since the state of the leaked gas is recorded as an image, there is an advantage that it can be used as evidence of the occurrence of gas leak and its repair.

International Publication No. 2016/143754 International Publication No. 2017/150565

However, since the equipment such as piping is complicatedly arranged in the gas equipment to be monitored by the gas leakage detection device, the location where the gas leakage occurs may be hidden behind the equipment and cannot be seen from the image pickup device. Can occur. In addition, when the wind direction is changing, it becomes difficult to identify the location where the gas leak occurs.

Therefore, by using machine learning to learn the gas image of the leaked gas and the feature amount related to the gas, for example, the gas from the inspection image under various conditions such as the position of the gas leak source and the gas flow velocity. When considering the realization of a method for identifying the features of a leaked gas cloud, it was necessary to efficiently collect a large amount of teacher data under various conditions regarding the leaked gas cloud.

This disclosure has been made in view of the above problems, and efficiently sets a large amount of training data consisting of input and correct output under various conditions for learning a machine learning model in a gas leak detection device. It is an object of the present invention to provide a data generation device for machine learning, a data generation method for machine learning, a program, and a data set for learning, which are collected and contribute to improvement of learning accuracy.

In the machine learning data generation method according to one aspect of the present disclosure, three-dimensional gas distribution image data showing the existence range of gas leaking from a gas leakage source into a three-dimensional space is observed from a predetermined viewpoint position2. It is characterized by including a step of converting into three-dimensional gas distribution image data and a step of generating the two-dimensional gas distribution image data and predetermined feature information about the gas as teacher data for machine learning.

According to the machine learning data generation device, the machine learning data generation method, the program, and the learning data set according to one aspect of the present disclosure, various conditions are used for learning the machine learning model in the gas leakage detection device. It is possible to efficiently generate a set of a large amount of learning data consisting of an input and a correct answer output in, and it is possible to contribute to improvement of learning accuracy.

It is a schematic block diagram of the gas leak detection system 1 which concerns on embodiment. It is a schematic diagram which shows the relationship between the monitoring object 300 and the gas visualization image pickup apparatus 10. It is a figure which shows the structure of the gas leak detection device 20. (A) is a functional block diagram of the control unit 21, and (b) is a schematic diagram showing an outline of the logical configuration of the machine learning model. It is a functional block diagram of the data generation apparatus 30 for machine learning. (A) is a functional block diagram of the machine learning data generation device 30, and (b) is a functional block diagram of the machine learning data generation device 30A according to the first modification. (A) and (b) are schematic diagrams showing the data structure of the structure three-dimensional data and the three-dimensional gas distribution image data, respectively. It is a schematic diagram for demonstrating the outline of the density-thickness product calculation method in two-dimensional single-viewpoint gas distribution image conversion processing. It is a schematic diagram for demonstrating the outline of the density-thickness product calculation method under the condition that a structure exists in a two-dimensional single-viewpoint gas distribution image conversion process. It is a flowchart which shows the outline of the 2D single-viewpoint gas distribution image conversion processing. (A) is a functional block diagram of the machine learning data generation device 30B according to the second embodiment, and (b) is a functional block diagram of the machine learning data generation device 30C according to the second modification. (A) is a schematic diagram showing an outline of density-thickness product image data, (b) is a schematic diagram for explaining an outline of a density-thickness product calculation method in a light absorption rate image conversion process, and (c) is light. It is a schematic diagram which shows the outline of the absorption rate image data. It is a functional block diagram of the machine learning data generation apparatus 30D which concerns on Embodiment 3. FIG. (A) is a schematic diagram showing an outline of the three-dimensional structure data, (b) is a schematic diagram showing an outline of the extracted background location data, and (c) is background image data based on the background location data. It is a conceptual diagram which showed the method of generating. (A) is a schematic diagram showing an outline of background image data, (b) is a schematic diagram showing an outline of light absorption rate image data, and (c) is a schematic diagram showing an outline of light intensity image data. It is a flowchart which shows the outline of the background image generation processing. It is a flowchart which shows the outline of the light intensity image data generation processing. (A) is a functional block diagram of the machine learning data generation device 30E according to the modification 3, and (b) is a functional block diagram of the machine learning data generation device 30F according to the modification 4. It is a functional block diagram of the machine learning data generation apparatus 30F which concerns on Embodiment 4. FIG. It is a flowchart which shows the outline of a gas distribution enhancement process. (A) is a functional block diagram of the machine learning data generation device 30G according to the modified example 5, and (b) is a functional block diagram of the machine learning data generating device 30H according to the modified example 6.

<< Embodiment 1 >>
<Configuration of gas leak detection system 1>
The embodiment of the present disclosure is realized as a gas leak detection system 1 that analyzes a gas leak from a gas leak inspection image of a gas facility. Hereinafter, the gas leak detection system 1 according to the embodiment will be described in detail with reference to the drawings.

FIG. 1 is a schematic configuration diagram of a gas leak detection system 1 according to an embodiment. As shown in FIG. 1, the gas leak detection system 1 includes a plurality of gas visualization image pickup devices 10 connected to a communication network N, a gas leak detection device 20, a machine learning data generation device 30, and a storage means 40. To.

The communication network N is, for example, the Internet, and a gas visualization image pickup device 10, a gas leak detection device 20, a plurality of machine learning data generation devices 30, and a storage means 40 are connected so as to be able to exchange information with each other. ..

(Gas visualization imager 10, etc.)
The gas visualization image pickup device 10 images the monitored object, processes the captured infrared image, colors the leaked gas portion in the inspection image, visualizes the gas leak portion of the inspection target, and displays the image on the gas leak detection device 20. The device or system provided. For example, an imaging means (not shown) consisting of an infrared camera that detects and captures infrared rays, a visualization processing means (not shown) that visualizes a gas leak portion of the inspection target from an inspection image captured by the imaging means, and a communication network. An interface circuit (not shown) that outputs to N is provided.

Images taken with an infrared camera are generally used for detecting hydrocarbon gases. For example, it is a so-called infrared camera that detects and images infrared light having a wavelength of 3.2 to 3.4 μm, and can detect hydrocarbon-based gases such as methane, ethane, ethylene, and propylene.

As shown in the schematic diagram of FIG. 2, the gas visualization image pickup device 10 is installed so that the monitoring target 300 is included in the field of view range 310 of the infrared camera. The obtained inspection image is, for example, a video signal for transmitting an image of 30 frames per second. The gas visualization image pickup device 10 converts the captured image into a predetermined video signal. In the present embodiment, the infrared image signal acquired from the infrared camera is restored to an image and output to the visualization processing means as a moving image composed of a plurality of frames. The image is an infrared photograph of the monitored object, and has the intensity of infrared rays as a pixel value.

If the size of the gas distribution image or the number of frames as a moving image is excessive, the amount of calculation for machine learning and judgment based on machine learning becomes large. In the first embodiment, the number of pixels of the gas distribution image is 224 × 224 pixels, and the number of frames is 16.

In the visualization processing means, the leaked gas portion in the inspection image is colored to visualize the gas leak to be inspected. For example, by executing the known optical gas leak detection method described in Patent Document 1 and the like, the function of the optical gas leak detection method is realized. Specifically, the gas visualization image pickup device detects the presence of gas by capturing a change in the amount of electromagnetic waves radiated from a background object having an absolute temperature of 0 (K) or higher. The change in the amount of electromagnetic waves is mainly caused by the absorption of electromagnetic waves in the infrared region by the gas or the generation of blackbody radiation from the gas itself. In the gas visualization image pickup device 10, gas leakage can be captured as an image by photographing the monitored space, so that gas can be captured earlier than the conventional detection probe type that can only monitor grid-like locations. Leakage can be detected and the location of gas can be accurately detected.

As described above, when there is a region where absorption or radiation is generated by the hydrocarbon gas in the infrared image of the inspection target, for example, the infrared change amount is converted into color information and the color information is mapped. The gas leak portion in the inspection image can be visualized with.

The visualized inspection image is temporarily stored in a memory or the like, and is transferred to and stored in the storage means 40 via the communication network N based on the operation input.

The gas visualization image pickup device 10 is not limited to this, and may be any image pickup device capable of detecting the gas to be monitored. For example, if the monitoring target is a gas that can be detected by visible light such as white smoked water vapor. It may be a general visible light camera. In the present specification, the gas refers to a gas leaked from a closed space such as a pipe or a tank, and is not intentionally diffused into the atmosphere.

Returning to FIG. 1, the storage means 40 is a storage device for storing inspection images transmitted from the gas visualization image pickup device 10, and is a volatile memory such as a DRAM (Dynamic Random Access Memory) and a non-volatile memory such as a hard disk. It is configured to include sexual memory.

(Gas leak detection device 20)
The gas leak detection device 20 is a device that acquires an image of a monitored object from the gas visualization image pickup device 10, detects a gas region based on the image, and notifies the user of the gas detection through the display unit 24. The gas leak detection device 20 is realized as, for example, a computer including a general CPU (Central Processing Unit), a RAM (Random Access Memory), and a program executed by these. As will be described later, the gas leak detection device 20 may further include a GPU (Graphics Processing Unit) and a RAM as arithmetic units.

Hereinafter, the configuration of the gas leak detection device 20 will be described. FIG. 3 is a diagram showing the configuration of the gas leak detection device 20. As shown in FIG. 3, the gas leak detection device 20 includes a control unit (CPU) 21, a communication unit 22, a storage unit 23, a display unit 24, and an operation input unit 25, and the control unit 21 executes a gas leak detection program. It is realized as a computer.

The communication unit 22 transmits / receives information to / from the gas leak detection device 20 and the storage means 40.

The display unit 24 is, for example, a liquid crystal panel or the like, and displays a display screen generated by the CPU 21.

The storage unit 23 stores the program 231 and the like required for the gas leak detection device 20 to operate, and also has a function as a temporary storage area for temporarily storing the calculation result of the CPU 21. The storage unit 23 includes, for example, a volatile memory such as a DRAM and a non-volatile memory such as a hard disk.

The control unit 21 realizes each function of the gas leak detection device 20 by executing the gas leak detection program 231 in the storage unit 23.

FIG. 4A is a functional block diagram of the control unit 21. FIG. 4B is a functional block diagram of the machine learning unit 2141 in the control unit 21.

As shown in FIG. 4A, the gas leak detection device 20 includes an inspection image acquisition unit 211, a gas distribution image acquisition unit 212, a leak position information acquisition unit 213, a machine learning unit 2141, a learning model holding unit 2142, and a determination result. The output unit 215 is provided. The machine learning unit 2141 and the learning model holding unit 2142 constitute a gas leak position identification unit 214. Further, the inspection image acquisition unit 211, the gas distribution image acquisition unit 212, the leak position information acquisition unit 213, the gas leak position identification unit 214, and the determination result output unit 215 constitute the gas leak position identification device 210.

The inspection image acquisition unit 211 is a circuit that acquires an inspection image from the gas visualization image pickup device 10. The inspection image is an image showing a gas distribution obtained by processing an infrared image captured by an infrared camera and coloring the leaked gas portion in the inspection image to visualize the gas leaked portion to be inspected.

The gas distribution image acquisition unit 212 is a circuit for acquiring a gas distribution image having the same format as the inspection image generated by the gas visualization image pickup device 10 and having a known gas leak position. Here, the gas distribution image is an image of an image of a gas cloud leaking from one leakage source, and there is only one gas leakage position, which is the position of the leakage source, with respect to the gas distribution image. If the acquired image is not in the same format as the gas distribution image generated by the inspection image acquisition unit 211, the gas distribution image acquisition unit 212 performs processing such as cutting out and gain adjustment so as to have the same format. You may go. Further, for example, when the acquired image is three-dimensional voxel data, it may be converted into a two-dimensional image of a viewpoint from one point.

The leak position information acquisition unit 213 is a circuit for acquiring the gas leakage position corresponding to the gas distribution image acquired by the gas distribution image acquisition unit 212. The gas leak position is designated as the coordinates in the gas distribution image acquired by the gas distribution image acquisition unit 212. If the acquired gas leak position is a coordinate in space, it is converted to the coordinate in the gas distribution image.

The machine learning unit 2141 executes machine learning based on the combination of the gas distribution image received by the gas distribution image acquisition unit 212 and the gas leakage position corresponding to the gas distribution image received by the leak position information acquisition unit 213. It is a circuit that generates a machine learning model. The machine learning model predicts the gas leak position indicating the position of the gas leak source based on the combination of the feature amount of the gas distribution image, for example, the outer peripheral shape of the gas region, the gas shading distribution, and the time change of these. Formed to do. As machine learning, for example, a convolutional neural network (CNN) can be used, and known software such as PyTorch can be used.

FIG. 4B is a schematic diagram showing an outline of the logical configuration of the machine learning model. The machine learning model includes an input layer 51, an intermediate layer 52-1, an intermediate layer 52-2, ..., An intermediate layer 52-n, and an output layer 53, and the interlayer filter is optimized by learning. For example, when the number of pixels of the gas distribution image is 224 × 224 pixels and the number of frames is 16, the input layer 51 accepts a 224 × 224 × 16 three-dimensional tensor in which the pixel value of the gas distribution image is input. The intermediate layer 52-1 is, for example, a convolution layer, and receives a 224 × 224 × 16 three-dimensional tensor generated by a convolution operation from the data of the input layer 51. The intermediate layer 52-2 is, for example, a pooling layer, and accepts a three-dimensional tensor obtained by resizing the data of the intermediate layer 52-1. The intermediate layer 52-n is, for example, a fully connected layer, and the data of the intermediate layer 52- (n-1) is converted into a two-dimensional vector showing coordinate values. The configuration of the intermediate layer is an example, and the number n of the intermediate layers is about 3 to 5, but the number n is not limited to this. Further, in FIG. 4B, the number of neurons in each layer is drawn as the same, but each layer may have an arbitrary number of neurons. The machine learning unit 2141 receives a moving image as a gas distribution image as an input, performs learning with the gas leakage position as the correct answer, generates a machine learning model, and outputs it to the learning model holding unit 2142. The machine learning unit 2141 may be realized by the GPU and software when the gas leak detection device 20 includes a GPU and a RAM as arithmetic units.

The learning model holding unit 2142 holds the machine learning model generated by the machine learning unit 2141 and outputs the gas leakage position corresponding to the gas distribution image acquired by the inspection image acquisition unit 211 using the machine learning model. Is. The gas leak position is specified and output as a coordinate value in the input gas distribution image.

The determination result output unit 215 is a circuit for generating an image for displaying on the display unit 24 by superimposing the gas leak position output by the learning model holding unit 2142 on the moving image acquired by the inspection image acquisition unit 211. ..

(Data generation device 30 for machine learning)
Hereinafter, the configuration of the machine learning data generation device 30 will be described. FIG. 5 is a diagram showing the configuration of the machine learning data generation device 30. As shown in FIG. 5, the machine learning data generation device 30 includes a control unit (CPU) 31, a communication unit 32, a storage unit 33, a display unit 34, and an operation input unit 35, and the control unit 31 provides machine learning data. It is realized as a computer that executes the generator.

The control unit 31 realizes the function of the machine learning data generation device 30 by executing the machine learning data generation program 331 in the storage unit 33.

FIG. 6A is a functional block diagram of the machine learning data generation device 30. FIG. 6B is a functional block diagram of the machine learning data generation device 30A according to the first modification, and the machine learning data generation device 30A will be described later. The condition parameters required for the processing input in each functional block in FIGS. 6A and 6B are as shown in the table below.

As shown in FIG. 6A, the machine learning data generation device 30 includes a three-dimensional structure modeling unit 311, a three-dimensional fluid simulation execution unit 312, and a two-dimensional single-viewpoint gas distribution image conversion processing unit 313.

The 3D structure modeling unit 311 designs a 3D structure unit model based on the operation input from the operator to the operation input unit 35, and performs 3D structure modeling for laying out the structure in the 3D space. The structure 3D data DTstr is output to the subsequent stage. The three-dimensional structure data DTstr is shape data representing, for example, the three-dimensional shape of a pipe or other plant facility. Commercially available 3D CAD (Computer-Aided Design) software can be used for 3D structure modeling.

FIG. 7A is a schematic diagram showing the data structure of the three-dimensional structure data DTstr. Here, in the present specification, the X direction, the Y direction, and the Z direction in each figure are the width direction, the depth direction, and the height direction, respectively. As shown in FIG. 7A, the structure three-dimensional data DTstr is three-dimensional boxel data representing a three-dimensional space, and is derived from the structure identification information Std arranged at the coordinates in the X direction, the Y direction, and the Z direction. It is composed. Since the structure identification information Std is expressed as three-dimensional shape data, it may be recorded as a binary image with 0 and 1 such as "with structure" and "without structure".

The 3D fluid simulation execution unit 312 acquires the structure 3D data DTstr as an input, and further acquires the condition parameter CP1 required for the fluid simulation based on the operation input from the operator to the operation input unit 35. The condition parameter CP1 is a setting required for fluid simulation mainly related to gas leakage, such as gas type, gas flow rate, gas flow velocity in three-dimensional space, shape, diameter, and position of gas leakage source, as shown in Table 1. It is a parameter that determines the conditions. By generating an image by changing many kinds of these conditional parameters, it is possible to generate a large number of training data.

Then, in the 3D space where the 3D structure modeling is performed, the 3D fluid simulation is performed, and the 3D gas distribution image data DTgas is generated and output to the subsequent stage. Three-dimensional gas distribution image data DTgas is data including at least three-dimensional gas concentration distribution. The calculation is performed using commercially available 3D fluid simulation software, and for example, ANSYS Fluent, Flo EFD, and Femap / Flow may be used.

FIG. 7B is a schematic diagram showing the data structure of the three-dimensional gas distribution image data DTgas. As shown in FIG. 7B, the three-dimensional gas distribution image data DTgas is three-dimensional voxel data representing a three-dimensional space, and voxel gas concentration data arranged at coordinates in the X, Y, and Z directions. It is composed of Dst (%) and may further include voxel gas flow velocity vector data Vc (Vx, Vy, Vz).

The two-dimensional single-viewpoint gas distribution image conversion processing unit 313 inputs and acquires the three-dimensional gas distribution image data DTgas of the gas leaking from the gas leakage source into the three-dimensional space, and further, to the operation input unit 35 from the operator. Based on the operation input of, the condition parameter CP2 necessary for the conversion process to the two-dimensional single-viewpoint gas distribution image is acquired. The condition parameter CP2 is a parameter related to the imaging conditions of the gas visualization image pickup device, such as the angle of view of the image pickup device, the line-of-sight direction, the distance, and the image resolution, as shown in Table 1, for example. Then, the two-dimensional single-viewpoint gas distribution image conversion processing unit 313 converts the three-dimensional gas distribution image data DTgas into the two-dimensional gas distribution image data observed from a predetermined viewpoint position. In the first embodiment, the process of converting the two-dimensional gas distribution image data into the density-thickness product image data DTdt is performed. The density thickness product image DTdt and the leak source coordinate data output from the three-dimensional structure modeling unit 311 are output to the gas leak detection device 20 as teacher data for machine learning.

The density-thickness product image DTdt is an image corresponding to an inspection image in which the leaked gas acquired by the gas visualization image pickup device 10 is captured, and is an image showing how the gas is seen from a viewpoint. Further, by considering the information of the three-dimensional structure data DTstr, it is possible to generate the gas concentration thickness product image data DTdt that is blocked by the structure and does not reflect the gas image that cannot be observed from the viewpoint.

FIG. 8 is a schematic diagram for explaining an outline of the concentration thickness product calculation method in the three-dimensional single-viewpoint gas distribution image conversion process, and is a gas concentration thickness from the three-dimensional gas distribution image data DTgas of gas showing the behavior of gas. It is a conceptual diagram of the method of generating the product image data DTdt.

The two-dimensional single-viewpoint gas distribution image conversion processing unit 313 spatially integrates the three-dimensional gas concentration image indicated by the three-dimensional gas distribution image data DTgas in the line-of-sight direction from a preset viewpoint position (X, Y, Z). A plurality of thickness product Dst values are generated by changing the angles θ and σ in the line-of-sight direction, and the obtained density thickness product Dst values are arranged two-dimensionally to generate density thickness product image data DTdt.

Specifically, as shown in FIG. 8, an arbitrary viewpoint position SP (X, Y, Z) is set in the three-dimensional space, and the three-dimensional gas distribution image data DTgas is set from the viewpoint position SP (X, Y, Z). The virtual image plane VF is set at a position separated by a predetermined distance in the direction of the three-dimensional gas concentration image indicated by. At this time, the virtual image plane VF is set so that the center O intersects the viewpoint position SP (X, Y, Z) and the straight line passing through the center voxel of the three-dimensional gas distribution image data DTgas. Further, the image frame of the virtual image plane VF is set according to the angle of view of the gas visualization image pickup apparatus 10. Then, the line-of-sight direction DA directed from the viewpoint position SP (X, Y, Z) toward the pixel A (x, y) of interest on the virtual image plane VF is the line-of-sight direction DO toward the central pixel O, that is, a gas visualization image pickup device. The angle θ is in the X direction, the angle is σ in the Y direction, and the direction is tilted with respect to the line-of-sight direction DO.

The gas concentration distribution data corresponding to the voxels of the three-dimensional gas distribution image that intersects the line of sight along the line-of-sight direction DA corresponding to the pixel A (x, y) of interest is spatially distributed in the line-of-sight direction DA with respect to the voxels that intersect the line of sight. By integrating, the value of the gas concentration thickness product for the pixel A (x, y) of interest is calculated. Then, while changing the angles θ and σ according to the angle of view of the gas visualization image pickup device 10, the positions of the pixels of interest A (x, y) are gradually moved, and all the pixels on the virtual image plane VF are the pixels of interest. By repeating the calculation of the value of the gas concentration thickness product as A (x, y), the density thickness product image data DTdt is calculated.

Furthermore, by using the same three-dimensional gas distribution image data DTgas and differentiating the viewpoint position SP (X, Y, Z) to generate the density thickness product image data DTdt, a plurality of concentration thicknesses can be obtained from one fluid simulation. Product image data DTdt can be easily generated.

FIG. 9 is a schematic diagram for explaining the outline of the concentration-thickness product calculation method under the condition that the structure exists in the three-dimensional single-viewpoint gas distribution image conversion process, and is a gas visualization image pickup device blocked by the structure. The case where it cannot be observed from 10 is shown.

In this case, in consideration of the three-dimensional position of the structure, the pixel A of interest is a method that does not perform spatial integration of the gas concentration distribution existing behind the structure when viewed from the viewpoint position SP (X, Y, Z). The value of the gas concentration thickness product is calculated for (x, y).

Specifically, the gas concentration thickness product for the pixel A (x, y) of interest is obtained by spatially integrating the viewpoint position SP (X, Y, Z) into the line-of-sight direction DA corresponding to the pixel A (x, y) of interest. Calculate the value of.

Then, by repeating the calculation of the value of the gas concentration thickness product in consideration of the three-dimensional position of the structure for all the pixels on the virtual image plane VF, the gas image that cannot be observed from the viewpoint because of being blocked by the structure is reflected. No gas concentration thickness product image DTdt is generated.

Returning to FIG. 5, the storage unit 33 stores the program 331 and the like necessary for the machine learning data generation device 30 to operate, and also serves as a temporary storage area for temporarily storing the calculation result of the control unit 31. Has the function of. The storage unit 33 includes a volatile memory such as a DRAM and a non-volatile memory such as a hard disk.

The communication unit 32 transmits / receives information to / from the machine learning data generation device 30 and the storage means 40.

The display unit 34 is, for example, a liquid crystal panel or the like, and displays a display screen generated by the CPU 31.

<Operation of two-dimensional single-viewpoint gas distribution image conversion processing>
Next, the two-dimensional single-viewpoint gas distribution image conversion processing operation in the machine learning data generation device 30 will be described with reference to the drawings.

FIG. 10 is a flowchart showing an outline of the two-dimensional single-viewpoint gas distribution image conversion process. The processing is executed by the two-dimensional single-viewpoint gas distribution image conversion processing unit 313 whose function is configured by the control unit 31.

First, the two-dimensional single-viewpoint gas distribution image conversion processing unit 313 acquires the structure three-dimensional data DTstr (step S1), and further acquires the three-dimensional gas distribution image data DTgas (step S2). Next, based on the operation input, as the condition parameter CP2, for example, the input of information regarding the angle of view of the image pickup device, the line-of-sight direction, the distance, and the image resolution is accepted (step S3). Further, based on the operation input, the viewpoint position SP (X, Y, Z) corresponding to the position of the image pickup portion of the gas visualization image pickup apparatus 10 is set in the three-dimensional space (step S4).

Next, a virtual image plane VF separated from the viewpoint position SP (X, Y, Z) in the direction of the three-dimensional gas concentration image by a predetermined distance is set, and as described above, the position of the image frame of the virtual image plane VF is visualized by gas. It is calculated according to the angle of view of the image pickup apparatus 10 (step S5).

Next, the coordinates of the pixel of interest A (x, y) are set to the initial value (step S6), and the pixel of interest A (x, y) on the virtual image plane VF is changed from the viewpoint position SP (X, Y, Z). The position LV on the line of sight is set to the initial value (step S7).

Next, it is determined whether or not the structure identification information Std of the boxel of the structure three-dimensional data DTstr that intersects the line of sight represents "no structure" (Std = 0) (step S8), and "structure". If "there is an object", the position of the pixel A (x, y) of interest is gradually moved (step S9), the process returns to step S7, and if it is not "there is a structure", the process proceeds to step S10.

In step S10, the presence or absence of voxels in the three-dimensional gas distribution image data DTgas that intersects the line of sight is determined, and if there is no voxel in the structure three-dimensional data DTstr that intersects the line of sight, the position LV on the line of sight is used as the unit length. (For example, increment by 1) (step S11), return to step S8, and if there is a voxel that intersects the line of sight, the density thickness value Dst of the voxel is read and the integration stored in the addition register or the like. The sum with the value Dsti is stored in an addition register or the like as a new integrated value Dsti (step S12).

Next, it is determined whether or not the calculation is completed for the total length of the line of sight corresponding to the range where the line of sight and the voxel intersect (step S13), and if not, the position LV on the line of sight is incremented by the unit length. Then (step S14), the process returns to step S8, and if completed, the voxels of the three-dimensional gas distribution image intersecting the line of sight along the line-of-sight direction DA corresponding to the pixel A (x, y) of interest. The gas concentration distribution data is spatially integrated to calculate the value of the gas concentration thickness product for the pixel A (x, y) of interest.

Next, it is determined whether or not the calculation of the gas concentration thickness product is completed for all the pixels on the virtual image plane VF (step S15), and if not, the pixel of interest A (x, y). ) Is gradually moved (step S16) and returned to step S7. If completed, the value of the gas concentration thickness product is calculated for all the pixels on the virtual image surface VF, and the virtual image surface VF is calculated. A gas concentration thickness product image DTdt is generated as the two-dimensional gas distribution image data.

Next, it is determined whether or not the generation of the gas concentration thickness product image DTdt is completed for all the viewpoint positions SP (X, Y, Z) to be calculated (step S17), and if not, the step is not completed. Returning to S4, the gas concentration thickness product image DTdt is generated for the new viewpoint position SP (X, Y, Z) input by the operation, and if it is completed, the process is terminated.

<Modification 1>
Although the machine learning data generation device 30 according to the first embodiment has been described above, the present disclosure is not limited to the above embodiments except for its essential characteristic components. Hereinafter, as an example of such an embodiment, a modified example of the above-described embodiment will be described.

FIG. 6B is a functional block diagram of the machine learning data generation device 30A according to the first modification. As shown in FIG. 6B, the machine learning data generation device 30A may be configured by removing the three-dimensional structure modeling unit 311 from the machine learning data generation device 30. In this case, the 3D fluid simulation execution unit 312 performs a 3D fluid simulation in a 3D space without a 3D structure and generates 3D gas distribution image data DTgas. Further, the two-dimensional single-viewpoint gas distribution image conversion processing unit 313 generates the density-thickness product image DTdt without considering the information of the structure three-dimensional data DTstr.

In the machine learning data generation device 30A, the structure 3D data DTstr is not generated, and the output gas concentration thickness product image data DTdt is in a state where the leakage source can be seen on the image. However, for example, after the gas concentration thickness product image data DTdt is generated, or when the gas concentration thickness product image data DTdt is generated, a masking process for concealing the leakage source portion in the gas concentration thickness product image is performed. , Gas concentration thickness product image data DTdt including masking information can be used as learning data. Here, too, by changing the condition parameters CP1 and CP2 and further changing the shape of the masking, a large number of various gas concentration thickness product image data DTdt can be created as learning data.

<Summary>
As described above, the various gas concentration thickness product image data DTdt generated by the machine learning data generation device 30 and the leakage source coordinate data PTlk are set and used as the machine learning teacher data used in the gas leakage detection device 20. Can be done.

Equipment such as piping is complicatedly arranged in the gas equipment to be monitored by the gas leak detection device 20. Assuming an application for constant monitoring, the image pickup device is installed at a specific location completely fixed at a specific location, or is installed at a specific location with only the shooting direction variable. In this case, the gas leak source may be hidden behind the equipment and cannot be seen from the image pickup apparatus.

Conventionally, there was a technique to identify the leak source position from the gas leak image, but in each case, since the leak source position is identified within the range visible from the image pickup device, there is a risk of presenting the wrong place as the leak source position. was there.

Also, if the wind direction is changing, it will be difficult to identify the location of the gas leak.

Therefore, using machine learning, we trained the gas image that captured the leaked gas and the correct answer data of the leak source position, and from the inspection image that captured the gas distribution, various conditions such as the position of the gas leak source and the gas flow velocity. It is conceivable to construct a method for identifying gas-related features using machine learning, which realizes a method for identifying gas-related features below.

However, machine learning generally requires 10,000 units of correct answer data, and in order to realize it, it is necessary to efficiently acquire a large amount of learning data for teachers under various conditions regarding leaked gas clouds. Become.

On the other hand, according to the machine learning data generation device and the machine learning data generation method according to the present embodiment, for learning the machine learning model in the gas leak detection device 20, for example, the gas leak image in this example. It is possible to efficiently generate tens of thousands of sets of learning data consisting of input and correct answer output, such as gas leakage source position coordinate information, and contribute to improving learning accuracy.

That is, by calculating image data corresponding to the gas leak image obtained by the image pickup device from the three-dimensional gas leak behavior simulation result obtained by the fluid simulation technology and using it as a learning data set for machine learning, the shooting conditions are taken. Since learning data can be obtained by freely changing various parameters that affect the behavior of gas leakage and gas leakage, as an example, when finding the gas leakage position from a gas leakage image using a machine learning model, under various conditions. A large amount of training data sets can be efficiently generated, which can contribute to improving learning accuracy.

<< Embodiment 2 >>
Next, the machine learning data generation device 30B according to the second embodiment will be described. FIG. 11A is a functional block diagram of the machine learning data generation device 30B according to the second embodiment. The same components as those of the machine learning data generation device 30 are assigned the same numbers, and the description thereof will be omitted.

The machine learning data generation device 30B is provided with a light absorption rate image conversion unit 314B after the two-dimensional single-viewpoint gas distribution image conversion processing unit 313, acquires the condition parameter CP3 based on the operation input, and obtains the gas concentration thickness product. The image data DTdt is further converted into the light absorption rate image data DTα and output to the subsequent stage.

12 (a) is a schematic diagram showing an outline of density-thickness product image data, FIG. 12 (b) is a schematic diagram for explaining an outline of a density-thickness product calculation method in a light absorption rate image conversion process, and FIG. 12 (c) is a schematic diagram. , It is a schematic diagram which shows the outline of the light absorption rate image data.

In the machine learning data generation device 30B, the value Dt of the gas concentration thickness product in the pixels (x, y) in the gas concentration thickness product image data DTdt shown in FIG. 12 (a) is as shown in FIG. 12 (b). Using the relationship between the gas concentration thickness product and the light absorption rate α, the value α of the light absorption rate corresponding to the gas concentration thickness product in the pixels (x, y) shown in FIG. 12 (c) is calculated. The light absorption rate α differs depending on the gas type specified by the condition parameter CP3, and the relationship data between the concentration thickness product value Dt and the light absorption rate α, for example, the gas concentration thickness product value Dt stored in a data table or the like. The gas concentration thickness product image data DTdt is converted into the light absorption rate image data Dtα based on the light absorption characteristics of the gas, based on the value of the light absorption rate α corresponding to Convert. The relationship between the value Dt of the gas concentration thickness product for each gas type and the light absorption rate α may be obtained in advance by actual measurement.

<Summary>
As described above, the light absorption rate image data DTα generated by the machine learning data generation device 30B and the leakage source coordinate data PTlk can be used as a set as the machine learning teacher data used in the gas leakage detection device 20.

As a result, in addition to the effect that a large amount of learning data can be efficiently generated by the machine learning data generation device 30A according to the first embodiment, the light closer to the gas distribution image obtained from the gas visualization image pickup device 10. By generating the absorption rate image data DTα as the teacher data, it is possible to generate the teacher data closer to the inspection image. As a result, it is possible to further improve the learning accuracy in the machine learning model in the gas leak detection device.

<Modification 2>
Further, as shown in FIG. 11B, as the second modification, the machine learning data generation device 30B is three-dimensionally similar to the machine learning data generation device 30A according to the modification 1 shown in FIG. 6B. The machine learning data generation device 30C excluding the structure modeling unit 311 may be configured. As a result, as in the first modification, the masking process for concealing the leakage source portion in the gas concentration thickness product image is performed by changing the masking shape in various ways, so that the light absorption rate image data including various masking information is included. DTα can be created as learning data.

<< Embodiment 3 >>
Next, the machine learning data generation device 30D according to the third embodiment will be described. FIG. 13 is a functional block diagram of the machine learning data generation device 30D according to the third embodiment. The same components as those of the machine learning data generation device 30B are numbered the same, and the description thereof will be omitted.

<Structure>
The machine learning data generation device 30D is newly provided with a background image generation unit 315D after the two-dimensional single-viewpoint gas distribution image conversion processing unit 313, and a new light intensity image is provided at the rear stage of the light absorption rate image conversion unit 314B. It differs from the machine learning data generation device 30B in that it includes a generation unit 316D.

The background image generation unit 315D acquires the background location data PTbk and the condition parameter CP5 based on the operation input, and generates the background image data DTIback.

FIG. 14A is a schematic showing an outline of the three-dimensional structure data. As shown in FIG. 14A, the structure three-dimensional data DTstr is three-dimensional voxel data representing a three-dimensional space, and the structure identification information Std is as three-dimensional shape data as in the first embodiment. Unlike the configuration that stores as a binary image such as "with structure" and "without structure", a value indicating the classification of the structure surface is given for each pixel, and a multi-value such as 0, 1, 2, 3, ... It is recorded as an image.

As described above, the structure identification information Std composed of this multi-valued image is subjected to a two-dimensional single-viewpoint conversion process using the virtual image plane VF observed from the viewpoint position SP (X, Y, Z) as an image frame. Background location data PTbk is generated.

FIG. 14B is a schematic diagram showing an outline of the extracted background location data PTbk. The background location data PTbk is two-dimensional image data composed of background classification Std (Std = 0, 1, 2, 3 ...) With respect to pixels A (x, y) in the virtual image plane VF. Here, the background classification Std is a classification number classified based on, for example, the optical characteristics of the surface of the structure. For example, the unpainted pipe may be set to 1, the painted pipe may be set to 2, and the concrete may be set to 3.

FIG. 14C is a conceptual diagram showing a method of generating background image data DTIback based on background location data PTbk. As shown in FIG. 14 (c), the background image data DTIback is generated based on the background location data PTbk generated as a multi-valued image and the condition parameter CP5 based on the operation input.

The condition parameter CP5 is a lighting condition for a structure such as a background two-dimensional temperature distribution, a background surface spectral emissivity, a background surface spectral reflectance, an illumination light wavelength distribution, a spectral illuminance, and an illumination angle, as shown in Table 1. , The temperature condition of the structure itself. When the background location data PTbk is a multi-valued image, the background image data DTIback is generated by adding a different condition parameter CP5 for each background classification Std. When the background location data is a binary image, the background image data DTIback is generated by giving different conditions for each background classification Std with and without the background.

The light intensity image generation unit 316D generates light intensity image data DTI based on the light absorption rate image data DTα, the background image data DTIback, and the gas temperature condition provided as the condition parameter CP4 based on the operation input.

15A is a schematic diagram showing an outline of background image data DTIback, FIG. 15B is a schematic diagram showing an outline of light absorption rate image data DTα, and FIG. 15C is a schematic diagram showing an outline of light intensity image data. It is a figure.

The infrared intensity at the coordinates (x, y) of the background image is DTIback (x, y), the blackbody radiation brightness corresponding to the gas temperature is Igas, and the light absorption rate at the coordinates (x, y) of the light absorption rate image. When the value of is DTα (x, y), the infrared intensity at the coordinates (x, y) of the light intensity image is calculated by Equation 1 as DTI (x, y).

The light intensity image generation unit 316D has coordinates (x, y) on the virtual image plane VF based on (Equation 1) based on the light absorption rate image data DTα, the background image data DTIback, and the gas temperature condition. ), The infrared intensity DTI (x, y) is calculated. As a result, the infrared radiation emitted by the background structure and the infrared rays emitted by the gas present in the voxels of the three-dimensional gas distribution image intersecting the line of sight along the line-of-sight direction DA corresponding to the pixel A (x, y) of interest. The total radiation can be calculated.

Then, the infrared intensity DTI (x, y) is calculated as all the pixels A (x, y) on the virtual image surface VF, and the light intensity image data DTI is generated.

<Operation of background image generation processing>
Next, the background image generation processing operation in the machine learning data generation device 30D will be described with reference to the drawings.

FIG. 16 is a flowchart showing an outline of the background image generation process. The process is executed by the background image generation unit 315D whose function is configured by the control unit 31.

First, the background image generation unit 315D acquires the structure three-dimensional data DTstr (step S1). The operation of steps S3 to S8 is the same as the operation of each step in FIG.

In step 8, when the background location data PTbk of the voxel intersecting the line of sight is “with a structure”, the background classification Std of the background location data PTbk is acquired (step S121A), and the condition parameter CP5 corresponding to the background classification Std is obtained. (Step S122A) is performed to determine the background image data value in the pixel A of interest, the position of the pixel A (x, y) of interest is gradually moved (step S9), and the process returns to step S7.

The condition parameter CP5 is, for example, a condition such as a background two-dimensional temperature distribution, a background surface spectral emissivity, a background surface spectral reflectance, an illumination light wavelength distribution, a spectral illuminance, and an illumination angle.

On the other hand, if it is not "with a structure", it is determined whether or not the calculation is completed for the total length of the line of sight corresponding to the range where the line of sight and the voxel intersect (step S13), and if not, the line of sight is not completed. The position LV on the line is incremented by the unit length (step S14), and the process returns to step S8. On the other hand, if it is completed, the standard value set when there is no structure is fixed as the background image data value in the pixel A of interest, and the calculation is completed for all the pixels on the virtual image surface VF. It is determined whether or not (step S15), and if it is not completed, the position of the pixel A (x, y) of interest is gradually moved (step S16), the process returns to step S7, and if it is completed, it is returned to step S7. End the process. Here, the standard value set when there is no structure is, for example, a background image data value corresponding to the ground or the sky in the real space. A standard value can be obtained by appropriately setting the conditions indicated by the condition parameter CP5.

As described above, the background classification Std of the background location data PTbk is acquired for all the pixels on the virtual image surface VF, and the background image data DTIback related to the virtual image surface VF is generated.

<Operation of light intensity image data generation processing>
Next, the light intensity image data generation processing operation in the machine learning data generation device 30D will be described with reference to the drawings.

FIG. 17 is a flowchart showing an outline of the light intensity image data generation process.

First, the background image data DTIback (x, y) relating to the virtual image plane VF and the light absorption rate image data DTα (x, y) are acquired (steps S101 and 102), and the condition parameter CP4 relating to the gas temperature condition is input (step). S103).

Next, the blackbody radiation luminance Igas corresponding to the gas temperature is acquired (step S104), the infrared intensity DTI (x, y) of the light intensity image is calculated by (Equation 1) (step S105), and the light intensity image is obtained. It is output as data DTI (step S106), and the process is terminated.

<Summary>
As described above, the light intensity image data DTI generated by the machine learning data generation device 30D and the leakage source coordinate data PTlk can be used as a set as machine learning teacher data used in the gas leakage detection device 20.

As a result, in addition to the effect that a large amount of learning data can be efficiently generated by the machine learning data generation devices 30 and 30B according to the first and second embodiments, the following effects are obtained. That is, by generating the light intensity image data DTI in a mode more similar to the gas distribution image obtained from the gas visualization image pickup device 10 as the teacher data, it is possible to generate the teacher data closer to the inspection image. The gas part can be easily extracted. As a result, it is possible to further improve the learning accuracy in the machine learning model in the gas leak detection device.

<Modifications 3 and 4>
FIG. 18A is a functional block diagram of the machine learning data generation device 30E according to the third modification. As shown in FIG. 18A, the background image generation unit 315E is different from the machine learning data generation device 30D in that the background image data DTIback is generated without using the background location data PTbk. In the background image generation unit 315E, for example, the background image data DTIback having uniform brightness may be generated regardless of the background location data PTbk.

FIG. 18B is a functional block diagram of the machine learning data generation device 30F according to the modified example 4. As shown in FIG. 18B, as a modification 4, a three-dimensional structure is formed from the machine learning data generation device 30D in the same manner as the machine learning data generation device 30A according to the modification 1 shown in FIG. 6B. The machine learning data generation device 30F excluding the modeling unit 311 may be configured. Thereby, as in the modifications 1 and 2, the gas concentration thickness product image data DTdt including various masking information can be created as learning data.

<< Embodiment 4 >>
Next, the machine learning data generation device 30G according to the fourth embodiment will be described. FIG. 19 is a functional block diagram of the machine learning data generation device 30G according to the fourth embodiment. The same components as those of the machine learning data generation device 30D are numbered the same, and the description thereof will be omitted.

<Structure>
The machine learning data generation device 30G is different from the machine learning data generation device 30D in that a gas distribution enhancement processing unit 317G is newly provided after the light intensity image generation unit 316D. The gas distribution enhancement process is an enhancement process for a predetermined frequency with respect to a time-series image so that the behavior of the gas distribution image diffused in space can be emphasized.

The gas distribution enhancement processing unit 317G generates gas distribution enhancement image data DTIem for the light intensity image data DTI by, for example, a known method described in International Publication No. 2017/0743430.

<Operation of gas distribution emphasized image data generation processing>
Next, the gas distribution-enhanced image data generation processing operation in the machine learning data generation device 30G will be described with reference to the drawings.

FIG. 20 is a flowchart showing an outline of the gas distribution emphasized image data generation process. First, for the time-series pixel data, a simple moving average with a predetermined number of frames as a unit is calculated to extract specific frequency component data (step S201), and the difference between the time-series pixel data and the specific frequency component data is obtained. Calculated as specific difference data (step S202).

Next, for the specific difference data, the moving standard deviation in units of a predetermined number of frames is calculated as the specific variation data (step S203), and the specific difference data or the specific variation data is used as the gas distribution-enhanced image data DTIem. Output (step S204) and end the process.

<Summary>
As described above, the gas distribution-enhanced image data DTIem generated by the machine learning data generation device 30G and the leakage source coordinate data PTlk can be used as a set as the machine learning teacher data used in the gas leakage detection device 20.

This has the following effects in addition to the effect that a large amount of learning data can be efficiently generated by the machine learning data generation devices 30, 30B, and 30D according to the first, second, and third embodiments. That is, the frequency of the gas portion in the gas distribution image obtained from the gas visualization image pickup apparatus 10 is extracted, and the gas distribution emphasized image data DTIem that emphasizes the gas portion is generated as teacher data to extract the target gas portion. You can easily do it. This can further contribute to the improvement of learning accuracy in the machine learning model in the gas leak detection device.

In addition, by making the frequency components to be extracted in the gas distribution image different, more diverse teacher data can be generated from the results of fluid simulation under the same conditions, and a large amount of training data can be obtained more efficiently. Can be generated.

<Modifications 5 and 6>
FIG. 21A is a functional block diagram of the machine learning data generation device 30H according to the modified example 5. Similar to the modification 3, the background image generation unit 315E is different from the machine learning data generation device 30G in that the background image data DTIback is generated without using the background location data PTbk. The background image generation unit 315E may generate, for example, background image data DTIback having uniform brightness.

FIG. 18B is a functional block diagram of the machine learning data generation device 30I according to the modified example 6. In the sixth modification, the machine learning data generation device 30I may be configured by removing the three-dimensional structure modeling unit 311 from the machine learning data generation device 30G. Thereby, as in the modified examples 1, 2 and 4, the gas concentration thickness product image data DTdt including various masking information can be created as learning data.

<Modification 7>
In the above-described embodiments 1 to 4, the two-dimensional gas distribution image obtained by observing the three-dimensional gas distribution image data showing the existence range of the gas leaking from the gas leakage source into the three-dimensional space from a predetermined viewpoint position. The data and the location information of the gas leak source are generated as teacher data for machine learning.

On the other hand, in the machine learning data generation device according to the modification 7, the two-dimensional gas distribution image data and the direction of the gas flow instead of the position information of the gas leak source are generated as teacher data for machine learning. This is different from the first to fourth embodiments. Except for the point that the direction of the gas flow is used as the teacher data, the same configuration as that of any of the first to fourth embodiments can be used, and the description thereof will be omitted.

In the description of FIG. 4B, as described above, the three-dimensional gas distribution image data may include voxel gas flow velocity vector data Vc (Vx, Vy, Vz).

In the machine learning data generation device according to the modification 7, a combination of the two-dimensional gas distribution image data generated based on the three-dimensional gas distribution image data and the parameter indicating the direction of the gas flow in the three-dimensional gas distribution image data is combined. Generated as teacher data and supplied to the machine learning department. ?? The parameter indicating the direction of the gas flow in the three-dimensional gas distribution image data can be calculated based on the gas flow velocity vector data Vc (Vx, Vy, Vz) of each voxel in the three-dimensional gas distribution image data.

Specifically, the gas flow velocity component in the line-of-sight direction DO when viewed from the viewpoint position SP is acquired from the gas flow velocity vector data Vc (Vx, Vy, Vz) of the voxel corresponding to the gas region in the three-dimensional gas distribution image data. .. The flow velocity component of the gas may be specified as a relative value with respect to the flow velocity component of the gas in the direction orthogonal to the line-of-sight direction DO. The direction orthogonal to the line-of-sight direction DO corresponds to the x-direction, y-direction, or the combined direction of the two-dimensional gas distribution image data, and is the direction in which the distance from the viewpoint position SP does not change. The flow velocity component of the gas in the direction orthogonal to the line-of-sight direction DO may be shown as a relative value with respect to the image size and the number of pixels of the gas distribution moving image.

Then, as a parameter indicating the direction of the gas flow, for example, the parameter is calculated for the two-dimensional gas distribution image data such as the average value and the maximum value of the flow velocity component of the gas.

The machine learning unit uses machine learning to perform learning using two-dimensional gas distribution image data that images leaked gas and parameters that indicate the direction of gas flow as correct data, and generates a machine learning model. Thereby, it is possible to construct a gas flow direction identification device and a method for identifying the gas flow direction by using machine learning from the inspection image obtained by imaging the gas distribution.

≪Other variants≫
Although the gas leak detection device according to the embodiment has been described above, the present disclosure is not limited to the above embodiment except for its essential characteristic components. For example, a form obtained by subjecting various modifications to the embodiment that a person skilled in the art can think of, or a form realized by arbitrarily combining the components and functions in each embodiment without departing from the spirit of the present invention. Is also included in this disclosure. Hereinafter, as an example of such an embodiment, a modified example of the above-described embodiment will be described.

(1) In the above-described embodiment, the gas visualization image pickup device 10 is configured to realize a function of coloring the leaked gas portion in the inspection image captured by the infrared camera to visualize the gas leaked portion to be inspected.

However, as a function of the gas leak detection device 20, a function of performing gas detection processing on a moving image output by an infrared camera to visualize a gas leak portion and generating a gas distribution image including a gas region is realized. May be good. As described above in the gas visualization image pickup apparatus 10, a known method can be used for the gas detection process. Specifically, for example, the method described in Patent Document 1 can be used. Then, a gas distribution image as a moving image obtained by cutting out a region including a gas region from each frame of the moving image is generated. When visualizing the gas leak portion, after cutting out the region including the gas region from each frame of the moving image, processing such as gain adjustment may be performed, or the pixel value of the moving image output by the infrared camera may be used. Instead, the difference in pixel values may be mapped to obtain a gas distribution image.

(2) In the above-described embodiment, a gas plant is illustrated as a gas facility as an example of an inspection image. However, the present disclosure is not limited to this, and may be applied to the generation of display images in equipment, devices, laboratories, laboratories, factories, and business establishments that use gas.

(3) In the above-described embodiment, an example of executing the processing of the optical gas leak detection method in the gas visualization image pickup apparatus 10 has been described. However, based on the original image of the inspection image stored in the storage means 40, the processing of the optical gas leak detection method is executed in the CPU 21 of the gas leak detection device 20, and the color information is mapped to the region where the gas component is detected. The inspection image with the mark may be transmitted from the gas leak detecting device 20 to the storage means 40 via the communication network N.

(4) Although the present disclosure has been described based on the above embodiment, the present disclosure is not limited to the above embodiment, and the following cases are also included in the present invention.

For example, the present invention is a computer system including a microprocessor and a memory, and the memory may store the computer program, and the microprocessor may operate according to the computer program. For example, it may be a computer system that has a computer program for processing in the gas leak detection system 1 of the present disclosure or a component thereof, and operates according to this program (or instructs each connected part to operate). ..

Further, in the present invention, all or part of the processing in the gas leak detection system 1 or its constituent elements is configured by a computer system composed of a microprocessor, a recording medium such as a ROM, a RAM, a hard disk unit, and the like. included. The RAM or the hard disk unit stores a computer program that achieves the same operation as each of the above devices. When the microprocessor operates according to the computer program, each device achieves its function.

Further, some or all of the components constituting each of the above devices may be composed of one system LSI (Large scale Integration). A system LSI is a super-multifunctional LSI manufactured by integrating a plurality of components on one chip, and specifically, is a computer system including a microprocessor, ROM, RAM, and the like. .. These may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them. A computer program that achieves the same operation as each of the above devices is stored in the RAM. When the microprocessor operates according to the computer program, the system LSI achieves its function. For example, the present invention also includes a case where the processing in the gas leak detection system 1 or its constituent elements is stored as an LSI program, and this LSI is inserted into a computer to execute a predetermined program (gas inspection management method). Is done.

The method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of the circuit cells inside the LSI may be used.

Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology.

Further, a part or all of the functions of the gas leak detection system 1 or its components according to each embodiment may be realized by executing a program by a processor such as a CPU. It may be a non-temporary computer-readable recording medium in which a program for performing the operation of the gas leak detection system 1 or its components is recorded. By recording a program or signal on a recording medium and transferring it, the program may be executed by another independent computer system. Needless to say, the above program can be distributed via a transmission medium such as the Internet.

Further, the gas leak detection system 1 or each component thereof according to the above embodiment may be configured to be realized by a programmable device such as a CPU, a GPU (Graphics Processing Unit) or a processor, and software. These components can be a single circuit component or an aggregate of a plurality of circuit components. Further, a plurality of components can be combined into one circuit component, or an aggregate of a plurality of circuit components can be formed.

(5) Division of functional blocks is an example, and even if multiple functional blocks are realized as one functional block, one functional block is divided into multiple, or some functions are transferred to other functional blocks. good. Further, the functions of a plurality of functional blocks having similar functions may be processed by a single hardware or software in parallel or in a time division manner.

Further, the order in which the above steps are executed is for exemplifying in order to specifically explain the present invention, and may be an order other than the above. Further, a part of the above steps may be executed simultaneously with other steps (parallel).

Further, at least a part of the functions of each embodiment and its modified examples may be combined. Further, the numbers used above are all exemplified for the purpose of specifically explaining the present invention, and the present invention is not limited to the exemplified numbers.

≪Summary≫
As described above, in the machine learning data generation method according to the embodiment, three-dimensional gas distribution image data showing the existence range of gas leaking from the gas leakage source into the three-dimensional space is observed from a predetermined viewpoint position. It is characterized by including a step of converting into two-dimensional gas distribution image data and a step of generating the two-dimensional gas distribution image data and predetermined feature information about the gas as teacher data for machine learning.

Further, in another aspect, in any of the above embodiments, the configuration may further include a step of generating the three-dimensional gas distribution image data based on the three-dimensional fluid simulation.

In another aspect, in any of the above embodiments, in the machine learning, two-dimensional gas distribution image data that is imaged from a predetermined viewpoint position and shows the existence range of the gas leaked into the space is input. It may be configured to construct a speculative model for identifying predetermined characteristic information regarding the gas.

Further, in another aspect, in any of the above embodiments, the step of modeling the three-dimensional structure in the three-dimensional space is further included, and in the three-dimensional fluid simulation, under the condition that the three-dimensional structure exists. It may be configured to calculate the three-dimensional gas distribution image in the three-dimensional space.

With this configuration, it is possible to efficiently generate a large amount of training data sets consisting of inputs and correct answer outputs under various conditions for learning machine learning models in gas leak detection devices, which contributes to improving learning accuracy. can.

In another aspect, in any of the above embodiments, the step of converting to the two-dimensional gas distribution image data is the viewpoint of the three-dimensional gas distribution image in the three-dimensional space indicated by the three-dimensional gas distribution image data. A plurality of values of the density thickness product spatially integrated in the line-of-sight direction from the position are generated by changing the angle in the line-of-sight direction, and the obtained density-thickness product values are arranged two-dimensionally to obtain the density-thickness product image data. The configuration may include a step of generating.

In another aspect, in any of the above embodiments, the step of converting to the two-dimensional gas distribution image data converts the concentration-thickness product value into a light absorption rate based on the light absorption characteristics of the gas and obtains light. The configuration may include a step of generating absorption rate distribution image data.

In another aspect, in any of the above embodiments, the step of converting to the two-dimensional gas distribution image data is based on the light absorption rate distribution image data and at least a temperature condition including a gas temperature condition. The configuration may include a step of calculating the light intensity distribution image data.

Further, in another aspect, in any of the above aspects, the predetermined feature information may be configured to be the position information of the leakage source or the direction of the gas flow.

Further, in another aspect, in any of the above embodiments, the two-dimensional gas distribution image data may be configured as a moving image showing a time-series change of the gas distribution.

Further, in the machine learning data generation method according to the embodiment, the three-dimensional gas distribution image data of the gas leaking from the gas leakage source into the three-dimensional space is converted into the two-dimensional gas distribution image data observed from a predetermined viewpoint position. It is also possible to include a two-dimensional single-viewpoint gas distribution image conversion processing unit and a teacher data generation unit that generates the two-dimensional gas distribution image data and predetermined feature information related to the gas as teacher data for machine learning. good.

In another aspect, in any of the above embodiments, in the machine learning, the two-dimensional gas distribution image data in which the existence range of the gas leaked into the space captured from a predetermined viewpoint position is shown as a gas region is obtained. As an input, it may be configured to construct a guess model that identifies predetermined characteristic information regarding the gas.

In another aspect, in any of the above embodiments, the step of converting to the two-dimensional gas distribution image data is the viewpoint of the three-dimensional gas distribution image in the three-dimensional space indicated by the three-dimensional gas distribution image data. A plurality of values of the density thickness product spatially integrated in the line-of-sight direction from the position are generated by changing the angle in the line-of-sight direction, and the obtained density-thickness product values are arranged two-dimensionally to obtain the density-thickness product image data. The configuration may include a step of generating.

Further, the program according to the embodiment is a program for causing a computer to perform machine learning data generation processing, and the machine learning data generation processing has a range of existence of gas leaking from a gas leakage source into a three-dimensional space. The shown 3D gas distribution image data is converted into 2D gas distribution image data observed from a predetermined viewpoint position, and the 2D gas distribution image data and predetermined feature information related to the gas are used for machine learning. It may be a program generated as teacher data of.

In another aspect, the two-dimensional gas distribution image data generated from the three-dimensional gas distribution image data showing the existence range of the gas leaking from the gas leakage source into the three-dimensional space, and the three-dimensional gas distribution image data. It is a learning data set for inferring the characteristic information by machine learning with the predetermined characteristic information about the gas contained in the above, and the three-dimensional gas distribution image data generated by the three-dimensional fluid simulation is It may be a learning data set including a plurality of data sets consisting of two-dimensional gas distribution image data converted into a two-dimensional image observed from a predetermined viewpoint position and the feature information corresponding to the two-dimensional gas distribution image data. ..

In another aspect, in any of the above embodiments, the predetermined feature information may be the position information of the gas leak source or the learning data set which is the direction of the gas flow.

In another aspect, in any of the above embodiments, a learning data set including the two-dimensional gas distribution image data that does not include information on the gas distribution at the leakage source may be configured.

In another aspect, in any of the above embodiments, the two-dimensional gas distribution image data may be configured as a learning data set which is a moving image showing a time-series change of the gas distribution.

≪Supplement≫
The embodiments described above all show a preferable specific example of the present invention. Numerical values, components, arrangement positions and connection forms of components, processing methods, processing orders, and the like shown in the embodiments are examples, and are not intended to limit the present invention. Further, among the components in the embodiment, those not described in the independent claims showing the highest level concept of the present invention will be described as arbitrary components constituting the more preferable form.

Further, the order in which the above method is executed is for exemplifying for concretely explaining the present invention, and may be an order other than the above. Moreover, a part of the above-mentioned method may be executed at the same time (parallel) with another method.

Further, for the sake of easy understanding of the invention, the scale of the components of each figure mentioned in each of the above embodiments may differ from the actual ones. Further, the present invention is not limited to the description of each of the above embodiments, and can be appropriately modified without departing from the gist of the present invention.

The machine learning data generation device, machine learning data generation method, program, and learning data set according to the embodiment of the present disclosure can be widely applied to a system that uses a gas leak in a gas facility for inspection.

1 Gas inspection management system 10 Gas visualization image pickup device 20 Gas leak detection device 21 Control unit (CPU)
210 Gas leak position identification device 211 Inspection image acquisition unit 212 Gas distribution image acquisition unit 213 Leakage source position acquisition unit 214 Gas leak source position identification unit 2141 Machine learning unit 2142 Learning model holding unit 215 Judgment result output unit 22 Communication unit 23 Storage unit 231 Program 24 Display unit 25 Operation input unit 30, 30A, 30B, 30C, 30D, 30E, 30F, 30G, 30H
Machine learning data generator 31 Control unit (CPU)
311 3D structure modeling unit 312 3D fluid simulation execution unit 313 2D single-viewpoint gas distribution image conversion processing unit 314B light absorption rate image conversion unit 315D background image generation unit 316D light intensity image generation unit 317G gas distribution enhancement processing unit 32 Communication unit 33 Storage unit 331 Program 34 Display unit 35 Operation input unit 40 Storage means

Claims (17)

1. A process of converting 3D gas distribution image data showing the existence range of gas leaking from a gas leakage source into a 3D space into 2D gas distribution image data observed from a predetermined viewpoint position.
   A machine learning data generation method including a step of generating the two-dimensional gas distribution image data and predetermined feature information related to the gas as teacher data for machine learning.
2. The machine learning data generation method according to claim 1, further comprising a step of generating the three-dimensional gas distribution image data based on a three-dimensional fluid simulation.
3. In the machine learning, a two-dimensional gas distribution image data that is imaged from a predetermined viewpoint position and shows the existence range of the gas leaked into the space is input, and a guess model for identifying a predetermined characteristic information about the gas is constructed. The machine learning data generation method according to claim 1 or 2.
4. Further, a step of modeling a three-dimensional structure in the three-dimensional space is included.
   The machine learning data generation method according to any one of claims 1 to 3, wherein in the three-dimensional fluid simulation, a three-dimensional gas distribution image in the three-dimensional space is calculated under the condition that the three-dimensional structure exists.
5. In the step of converting to the two-dimensional gas distribution image data, the value of the concentration thickness product obtained by spatially integrating the three-dimensional gas distribution image in the three-dimensional space indicated by the three-dimensional gas distribution image data in the line-of-sight direction from the viewpoint position is obtained. 1. Data generation method for machine learning described in.
6. The step of converting to the two-dimensional gas distribution image data includes the step of converting the concentration thickness product value into a light absorption rate based on the light absorption characteristics of the gas to generate the light absorption rate distribution image data according to claim 5. The described data generation method for machine learning.
7. The step of converting into the two-dimensional gas distribution image data includes the step of calculating the light intensity distribution image data based on the light absorption rate distribution image data and the temperature condition including at least the gas temperature condition according to claim 6. The described data generation method for machine learning.
8. The machine learning data generation method according to any one of claims 1 to 7, wherein the predetermined feature information is the position information of the leak source or the direction of the gas flow.
9. The two-dimensional gas distribution image data is a moving image showing a time-series change of the gas distribution.
   The machine learning data generation method according to any one of claims 1 to 8.
10. A two-dimensional single-viewpoint gas distribution image conversion processing unit that converts three-dimensional gas distribution image data of gas leaking from a gas leakage source into a three-dimensional space into two-dimensional gas distribution image data observed from a predetermined viewpoint position.
    A machine learning data generation device including a teacher data generation unit that generates the two-dimensional gas distribution image data and predetermined feature information related to the gas as teacher data for machine learning.
11. In the machine learning, a two-dimensional gas distribution image data in which the existence range of the gas leaked into the space, which is imaged from a predetermined viewpoint position, is shown as a gas region is input, and a guess model for identifying a predetermined characteristic information about the gas is specified. The machine learning data generation device according to claim 10.
12. In the step of converting to the two-dimensional gas distribution image data, the value of the concentration thickness product obtained by spatially integrating the three-dimensional gas distribution image in the three-dimensional space indicated by the three-dimensional gas distribution image data in the line-of-sight direction from the viewpoint position is obtained. The machine learning according to claim 10 or 11, further comprising a step of generating a plurality of values by changing the angle in the line-of-sight direction and arranging the obtained values of the density-thickness product in two dimensions to generate the density-thickness product image data. Data generator for.
13. A program that causes a computer to perform machine learning data generation processing.
    The machine learning data generation process is
    The 3D gas distribution image data showing the existence range of the gas leaking from the gas leakage source into the 3D space is converted into the 2D gas distribution image data observed from a predetermined viewpoint position.
    A program that generates the two-dimensional gas distribution image data and predetermined feature information about the gas as teacher data for machine learning.
14. The two-dimensional gas distribution image data generated from the three-dimensional gas distribution image data showing the existence range of the gas leaking from the gas leakage source into the three-dimensional space, and the predetermined gas included in the three-dimensional gas distribution image data. It is a learning data set for inferring the feature information by machine learning with the feature information of.
    Two-dimensional gas distribution image data obtained by converting three-dimensional gas distribution image data generated by a three-dimensional fluid simulation into a two-dimensional image observed from a predetermined viewpoint position, and
    A learning data set including a plurality of data sets consisting of the feature information corresponding to the two-dimensional gas distribution image data.
15. The learning data set according to claim 14, wherein the predetermined feature information is the position information of the gas leak source or the direction of the gas flow.
16. The learning data set according to claim 14, which includes the two-dimensional gas distribution image data that does not include information on the gas distribution at the leak source.
17. The learning data set according to any one of claims 14 to 16, wherein the two-dimensional gas distribution image data is a moving image showing a time-series change of the gas distribution.

Images