JP7445154B2 - Fin inspection system, fin inspection method, and program - Google Patents

Description

The present disclosure relates to a fin inspection system, a fin inspection method, and a program.

A visual inspection system for visually inspecting the fins of a heat exchanger is known. For example, an image processing method is known in which a fin-and-tube type heat exchanger is imaged with a camera to form a captured image of the fins, and a defective position is detected by removing a non-defective signal from the captured image (for example, patented (See Reference 1).

Japanese Patent Application Publication No. 2005-321300

In conventional techniques, if the defective location is small, defect information is also removed, so there is a problem in that it is difficult to detect a small defective location.

The present disclosure facilitates detection of small defects in a fin inspection system that performs visual inspection of fins of a heat exchanger.

A fin inspection system according to a first aspect of the present disclosure includes an image sensor that images a core of a heat exchanger, and a control unit that performs an external appearance inspection of fins used in the heat exchanger. The control unit uses machine learning to detect a fin defect from an image taken by the image sensor of the core of the heat exchanger, and outputs a detection result.

According to the first aspect of the present disclosure, in a fin inspection system that performs a visual inspection of fins of a heat exchanger, small defective spots can be easily detected.

A second aspect of the present disclosure is the fin inspection system according to the first aspect, in which the control unit detects fin defects by template matching on a partial image obtained by dividing the captured image into a plurality of regions. A certain possibility is evaluated, and a partial image evaluated as having a high possibility of fin defect is input to a machine learning model learned in advance to detect the fin defect. This makes it easier to detect small defective locations. In addition, since only partial images evaluated as having a high possibility of fin defects are input into the machine learning model, the inspection time can be shortened even for the core of a large heat exchanger.

A third aspect of the present disclosure is the fin inspection system according to the first aspect, wherein the control unit inputs partial images obtained by dividing the captured image into a plurality of regions into a pre-trained machine learning model. to detect the fin defect. This makes it easier to detect small defective locations.

A fourth aspect of the present disclosure is the fin inspection system according to the second or third aspect, wherein the machine learning model includes a partial image of a core of a heat exchanger and a fin defective in the partial image. This is a machine learning model trained in advance using training data indicating whether the

A fifth aspect of the present disclosure is the fin inspection system according to the second aspect, in which the control unit divides a partial image obtained by dividing the captured image into a plurality of regions and a defective template image prepared in advance. A degree of similarity is calculated, and a possibility that the partial image has a fin defect is evaluated based on the degree of similarity.

A sixth aspect of the present disclosure is the fin inspection system according to the fifth aspect, which includes a plurality of the defective template images, and the control unit is configured to control the fin inspection system according to the fifth aspect, wherein the control unit determines that the degree of similarity with any of the defective template images exceeds a threshold value. The partial images that exceed the above are input to the machine learning model.

A seventh aspect of the present disclosure is the fin inspection system according to any one of the first to sixth aspects, in which, when the fin defect is detected, the control unit includes the captured image and the fin inspection system. The detection result including information indicating the position where the defect was detected is output.

A fin inspection method according to an eighth aspect of the present disclosure is a fin inspection system including an image sensor that images a core of a heat exchanger, and a control unit that performs an external appearance inspection of fins used in the heat exchanger. The control unit uses machine learning to detect a fin defect from an image taken by the image sensor of the core of the heat exchanger, and outputs a detection result.

A program according to a ninth aspect of the present disclosure is a fin inspection system including an image sensor that images a core of a heat exchanger, and a control unit that performs an appearance inspection of fins used in the heat exchanger. The part detects a fin defect using machine learning from a captured image of the core of the heat exchanger taken by the image sensor, and outputs a detection result.

FIG. 1 is a diagram illustrating an example of a system configuration of a fin inspection system according to an embodiment. FIG. 2 is a diagram (1) showing a configuration example of a core of a heat exchanger according to an embodiment. FIG. 2 is a diagram (2) showing a configuration example of a core of a heat exchanger according to an embodiment. FIG. 3 is a diagram illustrating an example of arrangement of cameras according to an embodiment. FIG. 1 is a diagram illustrating an example of a hardware configuration of a fin inspection device according to an embodiment. 7 is a flowchart illustrating an example of fin inspection processing according to the first embodiment. FIG. 3 is a diagram for explaining fin inspection processing according to the first embodiment. 7 is a flowchart illustrating an example of template matching processing according to the first embodiment. FIG. 3 is a diagram for explaining a fin failure classification model according to the first embodiment. 7 is a flowchart illustrating an example of fin inspection processing according to the second embodiment.

Each embodiment will be described below with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, thereby omitting redundant explanation. Further, in this specification and the drawings, deviations in directions such as parallel, perpendicular, horizontal, perpendicular, up and down, and left and right are allowed to the extent that the effects of the present embodiment are not impaired. Moreover, the X-axis direction, the Y-axis direction, and the Z-axis direction include a direction parallel to the X-axis, a direction parallel to the Y-axis, and a direction parallel to the Z-axis, respectively.

<System configuration>
FIG. 1 is a diagram illustrating an example of a system configuration of a fin inspection system according to an embodiment. The fin inspection system 1 includes a camera (image sensor) 101 that captures an image of the core 10 of the heat exchanger, and a fin inspection device 100 that inspects the appearance of fins used in the heat exchanger. Preferably, the fin inspection system 1 includes a display device 102 that displays inspection results to an operator 103 or the like. Note that the fin inspection device 100 may display the inspection results on a display device 104 included in the fin inspection device 100.

For example, in the manufacturing process of a heat exchanger for an air conditioning system, the fin inspection system 1 images the core 10 of the heat exchanger with a camera 101, and uses machine learning from the captured image to inspect the fins constituting the core 10. This is a system that detects defects in products and outputs the detection results. For example, when a fin defect is detected, the fin inspection device 100 displays a captured image 31 of the core of the heat exchanger, information 32 indicating the position where the fin defect was detected, an enlarged image 33 of the defective location, etc. The display screen 30 of the defective location is displayed on the display device 102 (or display device 104) or the like.

2 and 3 are diagrams showing an example of the configuration of a core of a heat exchanger according to an embodiment. The heat exchanger according to the present embodiment is, for example, a fin-and-tube type heat exchanger, and includes a heat exchanger core 10 as shown in FIG. 2 .

The core 10 of the heat exchanger includes, for example, as shown in FIG. 2, a refrigerant pipe (tube) 210 and a plurality of fins 200 attached to the outer surface of the refrigerant pipe 210.

Each fin 200 is, for example, a member formed in a rectangular plate shape, and is arranged parallel to each other at regular intervals. In the example of FIG. 2, each fin 200 is arranged with its long side along the X-axis direction.

The refrigerant pipe 210 includes, for example, a plurality of straight pipe portions 211 that are straight circular pipes and curved pipe portions 212 that are circular pipes bent in a U-shape. In the example of FIG. 2, the refrigerant pipe 210 has straight pipe parts 211 and curved pipe parts 212 arranged alternately, and the straight pipe parts 211 are arranged along the Y-axis direction so as to be orthogonal to the fins 200. It is located.

FIG. 3 shows the core 10 of the heat exchanger as viewed from the direction of arrow 201 in FIG. As shown in FIG. 3, the straight pipe portion 211, which is a heat transfer tube, is provided so as to penetrate through each of the arranged fins 200. The outer circumferential surface of this straight pipe portion 211 is in close contact with each fin 200 and is thermally connected to each fin 200.

As a suitable example, the fin inspection system 1 according to the present embodiment aligns the fins 200 and performs an external appearance inspection of a plurality of fins before inserting the refrigerant pipes 210 into the aligned fins 200 in the process of manufacturing a heat exchanger. An example of this is a fin floating test. This is because if the aligned fins 200 are floating, the floating fins will be deformed when the refrigerant pipe 210 is inserted.

However, the present invention is not limited thereto, and the fin inspection system 1 may, for example, perform an external appearance inspection of the fins after inserting the refrigerant pipes 210 into the aligned fins 200. Furthermore, the fin inspection system 1 may be one that performs an external appearance inspection of the fins before the core 10 of the heat exchanger is assembled into a heat exchanger casing or the like. Here, as an example, the following explanation will be given assuming that the fin inspection system 1 performs a floating inspection of the aligned fins 200.

Note that the configuration of the core 10 of the heat exchanger shown in FIGS. 2 and 3 is an example. The core 10 of the heat exchanger may include a plurality of aligned fins 200, and the refrigerant tubes 210 may have other configurations.

The camera 101 is installed, for example, to image the core 10 of the heat exchanger from the direction of the arrow 301 in FIG. 3 (for example, from above the aligned fins 200). Note that when the size of the core 10 is large, the fin inspection system 1 uses a plurality of cameras 101-1 to 101-4 to image the core 10 of the heat exchanger, for example, as shown in FIG. It's okay. In this way, the number of cameras 101 included in the fin inspection system 1 may be any number greater than or equal to one.

<Hardware configuration>
FIG. 5 is a diagram illustrating an example of a hardware configuration of a fin inspection device according to an embodiment. The fin inspection device 100 has a general computer configuration, and includes, for example, a control unit 501, a memory 502, a storage device 503, a communication device 504, a display device 104, an input device 505, a drive device 506, and a bus 508. Including etc.

The control unit 501 is, for example, a processor such as a CPU (Central Processing Unit) that implements various functions by executing a predetermined program stored in a storage medium such as the storage device 503 or the memory 502. Note that the control unit 501 may include a processor such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) in addition to the CPU. Further, the control unit 501 may be, for example, a device such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The memory 502 includes, for example, RAM (Random Access Memory), which is a volatile memory used as a work area for the control unit 501, and ROM (Random Access Memory), which is a nonvolatile memory that stores a program for starting the control unit 501, etc. (Read Only Memory) etc. The storage device 503 is a large-capacity storage device that stores an OS (Operating System), programs such as applications, and various data and information, and is, for example, an SSD (Solid State Drive) or an HDD (Hard Disk Drive). It is realized by etc.

The communication device 504 is, for example, a communication interface or device for communicating with the camera 101, the display device 102, or the like. For example, the communication device 504 includes a device such as a NIC (Network Interface Card) for connecting the fin inspection device 100 to the communication network 2 and communicating with other devices. Further, the communication device 504 includes, for example, various interfaces for connecting the camera 101 to the fin inspection device 100.

The display device 104 is a display device or apparatus that displays a display screen. The input device 505 is, for example, an input device such as a touch panel, a keyboard, or a pointing device that receives input from the outside. Note that the display device 104 and the input device 505 may be an integrated display and input device, such as a touch panel display, for example.

The drive device 506 is a device for connecting the storage medium 507 to the fin inspection apparatus 100. The storage medium 507 herein includes, for example, a medium for recording information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk. Furthermore, the storage medium 507 may include, for example, a semiconductor memory that electrically records information, such as a ROM or a flash memory. A bus 508 is commonly connected to each of the above components, and transmits, for example, address signals, data signals, various control signals, and the like.

<Processing flow>
Next, the processing flow of the fin inspection method according to this embodiment will be explained.

[First embodiment]
FIG. 6 is a flowchart showing an example of fin inspection processing according to the first embodiment. This process shows an example of a fin inspection process executed by the fin inspection apparatus 100 having the hardware configuration shown in FIG. For example, the control unit 501 of the fin inspection apparatus 100 performs the fin inspection process shown in FIG. 6 by executing a predetermined program.

In step S601, the control unit 501 images the core 10 of the heat exchanger with the camera 101. For example, the control unit 501 acquires a captured image 710 of the core 10 of the heat exchanger, as shown in FIG. 7(A), from the camera 101. Note that, for example, as shown in FIG. 4, when capturing images of the core 10 of the heat exchanger with a plurality of cameras 101-1 to 101-4, the control unit 501 can capture images from the plurality of cameras 101-1 to 101-4, A captured image within the imaging range of each camera 101 is acquired.

As a preferred example, when performing a floating inspection of the aligned fins 200 in the process of manufacturing a heat exchanger, the control unit 501 checks the core 10 of the heat exchanger before inserting the refrigerant pipes 210 into the aligned fins 200. The aligned fins 200 are imaged.

In step S602, the control unit 501 projectively transforms the acquired captured image. For example, the control unit 501 performs known projective transformation on the image of the core 10 captured in the captured image 710 as shown in FIG. 7(A), thereby creating a rectangular shape as shown in FIG. An image 720 of the core 10 is generated. For example, the control unit 501 uses the OpenCV (registered trademark) library cv2. Projective transformation can be performed using a function such as warpPerspective(). However, the method of performing projective transformation is not limited to this.

Further, when the core 10 is imaged by a plurality of cameras 101-1 to 101-4, the control unit 501 projectively transforms the captured image obtained from each camera 101 into each region on one rectangle. An image 720 of the rectangular core 10 as shown in FIG. 7(B) is generated.

Note that the process in step S602 is optional. For example, the control unit 501 may omit the process of step S602 if the camera 101 can capture the image 720 of the rectangular core 10 due to the size of the core 10 or the position of the camera 101.

In step S603, the control unit 501 divides the image 720 of the core 10 into a plurality of partial images.

The fin inspection device 100 has a defective template image that is an image of the fin 200 in which a fin defect has occurred. For example, as shown in FIG. 7D, the fin inspection apparatus 100 prepares in advance a plurality of defective template images 741 and 742 of a predetermined size and stores them in a storage unit such as the storage device 503.

The control unit 501 converts the image 720 of the core 10 into a plurality of partial images 730 having the same size as the defective template images 741 and 742 or equivalent to the defective template images 741 and 742, for example, as shown in FIG. The image is divided into a plurality of partial images 730 of different sizes.

In step S604, the control unit 501 performs template matching of the divided partial images 730 with the defective template image to determine whether or not each partial image 730 has a high possibility of being a fin defect. For example, the control unit 501 executes template matching processing as shown in FIG.

FIG. 8 is a flowchart illustrating an example of template matching processing according to the first embodiment. This process shows an example of the template matching process executed by the control unit 501 in step S604 of FIG. 6, for example. For example, in step S604, the control unit 501 executes the process shown in FIG. 8 on each of the divided partial images 730.

In step S801, the control unit 501 calculates the degree of similarity between the partial image 730 and one or more defective template images. For example, the control unit 501 calculates the degree of similarity between the partial image 730 to be processed and a plurality of defective template images for each defective template.

For the similarity to be calculated, for example, a known similarity such as SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), NCC (Normalized Cross Correlation), or ZNCC (Zero Means Normalized Cross Correlation) may be applied. Can be done. However, the similarity calculated by the control unit 501 is not limited to these.

In step S802, the control unit 501 determines whether there is a defective template image whose calculated degree of similarity is greater than or equal to a threshold value. Here, it is assumed that the threshold is a value set in advance for determining that a fin defect has occurred when the degree of similarity exceeds the threshold.

If there is a defective template image with a degree of similarity equal to or greater than the threshold value, the control unit 501 moves the process to step S803. On the other hand, if there is no defective template image with a degree of similarity equal to or higher than the threshold value, the control unit 501 shifts the process to step S804.

In step S803, the control unit 501 determines that the partial image 730 to be processed is highly likely to have a fin defect. On the other hand, in step S804, the control unit 501 determines that the partial image 730 to be processed is unlikely to have a fin defect. The control unit 501 can extract partial images 730 with a high possibility of fin defects by executing the process shown in FIG. 8 on each of the divided partial images 730.

Now, returning to FIG. 6, the explanation of the flowchart of the fin inspection process will be continued. In step S605, the control unit 501 determines whether there is a partial image 730 with a high possibility of fin defect among the divided partial images 730. If there is no partial image 730 that is likely to be a fin defect, the control unit 501 moves the process to step S606. On the other hand, if there is a partial image 730 that is highly likely to be a fin defect, the control unit 501 shifts the process to step S607.

In step S606, the control unit 501 outputs an inspection result indicating that there is no fin defect. For example, the control unit 501 displays a display screen indicating that there is no fin defect in the core 10 of the heat exchanger on the display device 102, the display device 104, or the like.

On the other hand, in step S607, the control unit 501 inputs the partial image 730 that is likely to be a fin defect to a trained fin defect classification model that is stored in advance in a storage unit such as the storage device 503, for example.

FIG. 9 is a diagram for explaining the fin failure classification model according to the first embodiment. The fin failure classification model 900 is a machine learning model trained in advance using a plurality of partial images of the core 10 of the heat exchanger and teacher data indicating whether or not the partial image is a fin failure. For example, the fin defect classification model 900 inputs a large number of partial images (e.g., 1,000 or more) including partial images in which fin lifting occurs, by tagging them with "bad" or "good" training data. For example, machine learning using deep learning is being performed.

By inputting the partial image 730 into the trained fin defect classification model 900, the trained fin defect classification model 900 determines whether the input partial image 730 is "bad" or "good". Outputs the judgment result (classification result) indicating the Here, "defective" indicates that the fin is defective, and "good" indicates that there is no fin defect.

Note that the fin defect classification model 900 is an example of a machine learning model trained in advance using a partial image 730 of the core 10 of the heat exchanger and teacher data indicating whether the partial image 730 is a fin defect. It is.

Now, returning to FIG. 6 again, the explanation of the flowchart of the fin inspection process will be continued. In step S608, the control unit 501 determines whether there is any partial image 730 that was determined to have a fin defect in step S607. If there is no partial image 730 determined to be fin defective, the control unit 501 moves the process to step S606. On the other hand, if there is a partial image 730 determined to be defective in the fin, the control unit 501 moves the process to step S609.

In step S609, the control unit 501 outputs an image of the core 10 and inspection results indicating the position where the fin defect is detected. For example, the control unit 501 displays the display screen 30 of the defective location on the display device 102, the display device 104, etc. as shown in FIG.

Note that in the defective location display screen 30, the captured image 31 of the core 10 of the heat exchanger may be the captured image captured in step S601, or the image of the core 10 projectively transformed in step S602. It's okay. In addition, on the display screen 30 of the defective location, information 32 indicating the position where the fin defect was detected is, for example, the portion of the divided partial image 730 where the fin defect was determined, as shown in FIG. 7(C). It can be displayed using the position of the image 730. Further, on the display screen 30 of the defective part, the enlarged image 33 of the defective part is created using a partial image 730 determined to be defective among the divided partial images 730, for example, as shown in FIG. 7(C). can be displayed.

As described above, according to the first embodiment, the fin inspection system 1 that performs the visual inspection of the fins 200 of a heat exchanger can easily detect small defective parts.

In addition, in the first embodiment, only the partial image 730 that is evaluated as having a high possibility of being defective is input to the machine learning model (fin defect classification model 900). It can also shorten inspection time.

[Second embodiment]
The fin inspection apparatus 100 can also omit the template matching process described in FIG. 8 and perform an external appearance inspection of the fin 200. For example, when the size of the core 10 of the heat exchanger is relatively small, or when the defective template image cannot be prepared in time, the fin inspection apparatus 100 may omit the template matching process.

FIG. 10 is a flowchart illustrating an example of fin inspection processing according to the second embodiment. This process shows another example of the fin inspection process executed by the fin inspection apparatus 100 having the hardware configuration shown in FIG. For example, the control unit 501 of the fin inspection apparatus 100 performs the fin inspection process shown in FIG. 10 by executing a predetermined program. Note that the basic processing content is the same as the fin inspection processing according to the first embodiment described in FIG. 6, so detailed explanation of the processing content similar to the first embodiment will be omitted here.

In step S1001, the control unit 501 images the core 10 of the heat exchanger with the camera 101. For example, the control unit 501 acquires a captured image 710 of the core 10 of the heat exchanger, as shown in FIG. 7(A), from the camera 101.

In step S1002, the control unit 501 projectively transforms the acquired captured image. For example, the control unit 501 performs known projective transformation on the image of the core 10 captured in the captured image 710 as shown in FIG. 7(A), thereby creating a rectangular shape as shown in FIG. An image 720 of the core 10 is generated.

In step S1003, the control unit 501 divides the image 720 of the core 10 into a plurality of partial images. For example, the control unit 501 divides the image 720 of the core 10 into a plurality of partial images 730, as shown in FIG. 8(C).

In step S1004, the control unit 501 inputs each divided partial image 730 to a learned fin failure classification model stored in advance in a storage unit such as the storage device 503, for example. As a result, a determination result (classification result) indicating whether each input partial image 730 is "bad" with a fin defect or "good" with no fin defect is obtained.

In step S1005, the control unit 501 determines whether there is any partial image 730 that was determined to have a fin defect in step S1004. If there is a partial image 730 determined to be defective in the fin, the control unit 501 moves the process to step S1006. On the other hand, if there is no partial image 730 determined to be defective in the fin, the control unit 501 shifts the process to step S1007.

In step S1006, the control unit 501 outputs an image of the core 10 and inspection results indicating the position where the fin defect is detected. For example, the control unit 501 displays the display screen 30 of the defective location on the display device 102, the display device 104, etc. as shown in FIG.

On the other hand, in step S1007, the control unit 501 outputs an inspection result indicating that there is no fin defect. For example, the control unit 501 displays a display screen indicating that there is no fin defect in the core 10 of the heat exchanger on the display device 102, the display device 104, or the like.

The process shown in FIG. 10 facilitates the detection of small defective parts in the fin inspection system 1 that performs the visual inspection of the fins 200 of the heat exchanger, as in the first embodiment.

As described above, according to each embodiment of the present disclosure, small defective spots can be easily detected in the fin inspection system 1 that performs an external inspection of the fins 200 of a heat exchanger.

For example, in the conventional technique disclosed in Patent Document 1, a defective position is detected by removing a non-defective signal from a captured image. Therefore, in the conventional technology, if the defective location is small, the defect information is also removed, so there is a problem that it is difficult to detect the small defective location.

On the other hand, the fin inspection system 1 according to each embodiment of the present disclosure does not perform the process of removing the non-defective signal from the captured image, and therefore does not remove defective information. In addition, the fin inspection system 1 divides the image 720 of the core 10 of the heat exchanger into a plurality of partial images 730, inputs them into a fin defect classification model 900 trained in advance using the plurality of partial images and training data, and Determine whether or not there is a defect. Therefore, the fin inspection system 1 can easily detect even small defective parts.

Although the embodiments have been described above, it will be understood that various changes in form and details can be made without departing from the spirit and scope of the claims.

1 Fin inspection system 10 Heat exchanger core 30 Defective location display screen (example of inspection results)
101 Camera (image sensor)
200 Fin 501 Control unit 710 Captured image 730 Partial image 741, 742 Defective template image 900 Fin defect classification model (an example of machine learning model)

Claims (6)

A fin inspection system comprising an image sensor that images a core of a heat exchanger, and a control unit that performs an external appearance inspection of fins used in the heat exchanger,
The control unit includes:
Calculating the degree of similarity between a partial image obtained by dividing a captured image of the core of the heat exchanger into a plurality of regions and a defective template image prepared in advance by the image sensor,
inputting the partial image whose similarity with the defective template image exceeds a threshold into a pre-trained machine learning model to detect a fin defect, and outputting a detection result;
Fin inspection system. 2. The machine learning model is a machine learning model trained in advance using a partial image of the core of the heat exchanger and teacher data indicating whether the partial image has a fin defect. fin inspection system. having a plurality of the defective template images;
The fin inspection system according to claim 1 or 2 , wherein the control unit inputs the partial image whose similarity with any of the defective template images exceeds a threshold value to the machine learning model. Any one of claims 1 to 3, wherein, when the fin defect is detected, the control unit outputs the detection result including the captured image and information indicating a position where the fin defect is detected. Fin inspection system described in. A fin inspection system comprising an image sensor that images a core of a heat exchanger, and a control unit that performs an appearance inspection of fins used in the heat exchanger,
The control section,
Calculating the degree of similarity between a partial image obtained by dividing a captured image of the core of the heat exchanger into a plurality of regions and a defective template image prepared in advance by the image sensor,
inputting the partial image whose similarity with the defective template image exceeds a threshold into a pre-trained machine learning model to detect a fin defect, and outputting a detection result;
Fin inspection method. A fin inspection system comprising an image sensor that images a core of a heat exchanger, and a control unit that performs an appearance inspection of fins used in the heat exchanger,
In the control section,
Calculating the degree of similarity between a partial image obtained by dividing a captured image of the core of the heat exchanger into a plurality of regions and a defective template image prepared in advance by the image sensor;
inputting the partial image whose similarity to the defective template image exceeds a threshold into a pre-trained machine learning model to detect a fin defect and outputting a detection result;
program.

Images