JP7075331B2 - Water leak inspection system - Google Patents

Description

The present invention relates to a water leak inspection system.

Devices such as washing machines, dehumidifiers, humidifiers, water heaters, and floor heaters that use water inside them can cause enormous damage if water leaks. Water leak inspection is essential at the time of manufacture or shipment of these. On the other hand, since there are usually a large number of devices to be inspected, it is preferable that the water leak inspection can be performed automatically. As a method of such a water leak inspection, it is conceivable to detect a water leak by acquiring an image of the device to be inspected and determining the image.

However, when detecting a water leak using an image, it is necessary to prevent a so-called erroneous detection in which a phenomenon other than the water leak is detected as a water leak. As a method of preventing such false detection, it is conceivable to generate mask data for excluding the falsely detected part (the part where there is no water leakage) from the inspection target, and perform image judgment based on this. Be done. As a technique related to image masking, for example, Patent Document 1 discloses a method of generating a mask when inspecting defects such as print omissions using image density information.

Japanese Patent Application Laid-Open No. 2003-057190

However, it is considered unrealistic to apply the method as in Patent Document 1 to this case. That is, although Patent Document 1 assumes masking of printed matter, a device such as the washing machine described above has a complicated three-dimensional shape unlike printed matter, and is made of various materials. For example, for a device such as a washing machine, a synthetic resin is used for the housing, and various metal parts and materials such as rubber are also used. Further, since such a device may be inspected while being operated with water, vibration, movement, and deformation of the housing occur with the operation of this device. When an image is taken with respect to a device in such a state, an image like water may be acquired even if it is not water due to reflection of light rays or the like, which may lead to erroneous detection.

On the other hand, in order to prevent such false detection, it is conceivable to manually generate the mask without using the method as in Patent Document 1, but when there are many devices to be inspected, a user input error can be considered. There is a high possibility that mask data will be created by mistake due to such factors. Furthermore, in the case of inspection using images in this way, it is desirable to confirm in advance that the images can be taken properly, but it is very complicated to do this manually, and inspection mistakes (water droplets) It may cause false detection).

The present invention has been made in view of such a situation, and an object of the present invention is to provide a water leak inspection system for accurately detecting a water leak from an object to be inspected.

One of the present inventions for solving the above problems is a water leakage inspection system for determining water leakage from the surface of an object to be inspected capable of containing water, and the water leakage inspection system is a visible light camera. And an information processing device communicably connected to the visible light camera, the information processing device has substantially the same structure as the object to be inspected from the visible light camera in the visible light learning mode. , The first visible light image of the surface of the structure in which water leakage does not occur on the surface is acquired at a plurality of timings, and the gradation difference is extended with the progress of time based on each acquired first visible light image. The characteristic region of the first visible light image is specified, the identified characteristic region and its peripheral region are set as the visible light mask region, and water is contained in the visible light inspection mode. The second visible light image of the surface of the object to be inspected is acquired at a plurality of timings, and each of the acquired second visible light images excluding the portion corresponding to the visible light mask region is used as a visible light determination region. The water is contained when a characteristic region of the visible light determination region in which the gradation difference extends with the progress of time is specified based on each specified and identified visible light determination region. It is determined that water leakage has occurred in the object to be inspected.

According to the present invention, water leakage from the object to be inspected can be accurately detected.

FIG. 1 is a diagram showing an example of the configuration of the water leak inspection system 1 according to the first embodiment. FIG. 2 is a diagram showing an example of an installation method of the visible light camera 20. FIG. 3 is a diagram showing an example of an installation method of the far-infrared camera 50. FIG. 4 is a diagram showing an example of the configuration of the image processing device 200. FIG. 5 is a diagram showing an example of a menu screen on which the image processing device 200 is displayed when the image processing device 200 is started. FIG. 6 is a flow chart illustrating an example of the automatic inspection process. FIG. 7 is a flow chart illustrating an example of the visible light image pre-inspection process. FIG. 8 is a flow chart illustrating the details of the pre-water droplet determination process. FIG. 9 is a flow chart illustrating an example of the far-infrared image pre-inspection process. FIG. 10 is a flow chart illustrating the details of the pre-residual moisture determination process. FIG. 11 is a flow chart illustrating an example of visible light learning processing. FIG. 12 is a flow chart illustrating the details of the visible light mask data generation process. FIG. 13 is a flow chart illustrating an example of far-infrared learning processing. FIG. 14 is a flow chart illustrating the details of the far-infrared mask data generation process. FIG. 15 is a flow chart illustrating an example of visible light image determination processing. FIG. 16 is a flow chart illustrating the details of the visible light image mask processing. FIG. 17 is a flow chart illustrating an example of far-infrared image determination processing. FIG. 18 is a flow chart illustrating the details of the far-infrared image mask processing. FIG. 19 is a diagram showing an example of a learning mode screen. FIG. 20 is a diagram showing an example of a screen during creation of visible light mask data. FIG. 21 is a diagram showing an example of a screen during creation of far-infrared mask data. FIG. 22 is a diagram showing an example of a water leak inspection result display screen. FIG. 23 is a diagram showing an example of the configuration of the image processing device 200 according to the second embodiment. FIG. 24 is a diagram showing an example of error information 260. FIG. 25 is a flow chart illustrating an example of the visible light mask data generation process according to the second embodiment. FIG. 26 is a flow chart illustrating an example of the far-infrared mask data generation process according to the second embodiment.

The water leak inspection system of this embodiment will be described with reference to the drawings.
[First Embodiment]
<< Configuration of water leak inspection system >>
FIG. 1 is a diagram showing an example of the configuration of the water leak inspection system 1 according to the first embodiment. The water leak inspection system 1 is introduced into an inspection area 100 for performing a water leak inspection of one or a plurality of objects to be inspected 5 which are sequentially brought in from the outside. As a result of performing a water leak inspection on each inspected object 5, the inspected object 5 determined to have no water leakage is carried out of the inspection area 100. On the other hand, if it is determined that the inspected object 5 has a water leak, that fact is notified and appropriate treatment (repair, re-inspection, disposal, etc.) is performed.

The object to be inspected 5 is a device that accommodates water inside and performs a predetermined operation, and is provided with a flow path such as a pipe through which water flows. The water leak inspection system 1 inspects whether or not water leakage has occurred from the surface of the inspected object 5 or the surface of the internal piping thereof during the operation of the inspected object 5. The object 5 to be inspected is, for example, a washing machine, a dehumidifier, a humidifier, and a water heater. In the present embodiment, the object to be inspected 5 is a washing machine.

The inspection area 100 has a first inspection area 120 and a second inspection area 140, which are spaces that are shielded or insulated by a predetermined light-shielding material or heat insulating material (for example, a resin sheet).

As shown in FIG. 1, a transport device 160 is provided in the first inspection area 120 and the second inspection area 140. The transport device 160 is, for example, a roller conveyor or a belt conveyor, and moves the object 5 to be inspected at a set speed to pass through the inspection area 100, respectively, in the first inspection area 120 and the second inspection area 140. It is a device that allows water leak inspection. The transport device 160 transports the inspected object 5 arranged at the entrance 122 of the first inspection area 120 to the exit 124 of the first inspection area 120, and subsequently from the entrance 142 of the second inspection area 140 to the second inspection area. The inspected object 5 is carried to the exit 144 of 140.

At least one visible light camera 20 (visible light image acquisition device), a dehumidifying device 30, and a blower device 40 are installed in the first inspection area 120.

Further, the first inspection area 120 is provided with a first dropping system 400 for a preliminary inspection (described later) regarding a visible light image. The first dropping system 400 is provided at a water supply device 401 (pump or the like) for delivering water, a pipe 412 through which water flows from the water supply device 401, and a tip of each pipe 412, and drops water from each pipe 412. It is configured to include a nozzle 402 for dropping the water.

The visible light camera 20 acquires a visible light image of the object 5 to be inspected 5 at a plurality of timings in the first inspection area 120 by the transport device 160 (note that the visible light image may be a still image or a part of a moving image). The visible light camera 20 is fixed at a predetermined position.

The visible light camera 20 collects visible light reflected on the surface of the object 5 to be inspected or the surface of the internal piping thereof, and generates an image including shading information on these surfaces. In the present embodiment, the visible light camera 20 is a camera (camera with LED illumination) having a function of emitting visible light to the object 5 to be inspected by an LED (Light Emitting Diode) at the time of shooting to generate a gray scale image. do.

Here, FIG. 2 is a diagram showing an example of an installation method of the visible light camera 20. The visible light camera 20 leaks from the inspected object 5 which descends from the surface of the inspected object 5 or the surface of the internal pipe 5f or the like, or falls from the surface of the inspected object 5 or the surface of the internal pipe or the like. Take a picture of the water drops. For example, the visible light camera 20a is set to photograph the surface of the clothing inlet 5a, which is particularly prone to water leakage, and the surface 5b below the surface of the front surface of the object 5 to be inspected. The shooting direction of the visible light camera 20b is set so as to shoot the back surface 5c of the washing machine. The shooting direction of the visible light camera 20c is set so as to shoot the lower 6 of the washing machine. The distance between the object to be inspected 5 and each visible light camera 20 is, for example, about several tens of cm.

Further, each nozzle 402 is fixed at a predetermined position so that the water droplet 7 falling from the nozzle 402 falls within the shooting range of the visible light camera 20. For example, the position of each nozzle 402 is such that the falling water droplet 7 is about halfway between the visible light camera 20 and the object 5 to be inspected.

Since the water droplet 7 falling from the nozzle 402 is assumed to be a water droplet from the object to be inspected 5 detected in the first inspection area 120, it is preferable that the water droplet 7 has such a size.

Next, the dehumidifying device 30 of the first inspection area 120 shown in FIG. 1 is a device that reduces the humidity of the space in which the object to be inspected 5 exists (humidity of the first inspection area 120). The blower device 40 is a device that blows air toward the surface of the object to be inspected 5 moving in the first inspection area 120 by the transport device 160 and the surface of the internal piping or the like. As will be described later, the dehumidifying device 30 and the blowing device 40 are pretreatment devices for water leakage inspection of the object to be inspected 5 performed in the second inspection area 140. As will be described later, the pretreatment device is provided to promote the drying of the surface of the object to be inspected 5 and the surface of the internal piping and the like, and to improve the accuracy of the water leakage inspection in the second inspection area 140.

Next, in the second inspection area 140, a far-infrared camera 50 (far-infrared image acquisition device), a dehumidifying device 30, a blower device 40, and a temperature measuring device 80 are installed.

Further, the second inspection area 140 is provided with a second dropping system 450 for a preliminary inspection (described later) regarding a far-infrared image. The second dripping system 450 is provided at a water supply device 405 (pump or the like) for delivering water, a pipe 413 through which water from the water supply device 405 flows, and a tip of each pipe 413, and fine water droplets (fine water droplets (pumps, etc.) are provided from each pipe 413. For example, it is configured to include a nozzle 403 for dropping (water droplets smaller in size than the water droplets falling in the first inspection area 120) and a test plate 404 which is a member for receiving the fine water droplets dropped from the nozzle 403.

The far-infrared camera 50 acquires a far-infrared image of the object 5 to be inspected in the second inspection area 140 by the transport device 160. That is, the far-infrared camera 50 is the inspected object 5 in which the surface of the first inspection area 120 and the surface of the internal piping and the like are dried (a part of the moisture leaked to the surface is vaporized). The far-infrared image of the surface and the surface of the internal piping etc. is acquired. Like the visible light image, this far-infrared image is an image including shade information such as gray scale. The far-infrared camera 50 is attached to the articulated robot 55 as described later.

Here, FIG. 3 is a diagram showing an example of an installation method of the far-infrared camera 50. Since the far-infrared camera 50 of the present embodiment is used to detect residual water finer than the water droplets detected in the first inspection area 120, the shooting range of the far-infrared camera 50 is set narrow (for example, shooting). The range is a rectangular range with a side of about 10 to 20 cm).

Therefore, the far-infrared camera 50 of the present embodiment quickly performs imaging, and in order to ensure that the surface of the object 5 to be inspected and the surface of the internal piping and the like are photographed while the object 5 is moving in the second inspection area 140, many images are taken. It is attached to the joint robot 55. The articulated robot 55 can freely change its position and posture autonomously or by the image processing device 200 described later. As a result, the far-infrared camera 50 can quickly acquire a far-infrared image of a desired range on the surface of the object 5 to be inspected and the surface of the internal piping or the like.

Specifically, as shown in FIG. 3, the far-infrared camera 50a photographs the back surface 5c of the washing machine. Further, the far-infrared camera 50b photographs the clothes inlet 5a on the front surface of the washing machine, the lower surface 5b thereof, and the lower 6 of the washing machine. Further, in the present embodiment, the far-infrared camera 50a photographs the bottom surface 5d of the washing machine from below the washing machine by moving the articulated robot 55 to the lower part 6 of the washing machine. The far-infrared camera 50a also photographs the internal pipe 5f of the object 5 to be inspected.

The number of far-infrared cameras 50 can be appropriately changed depending on the inspection time for the object to be inspected 5 and the like. Further, since the far-infrared camera 50 has a weak waterproof property, a waterproof cover may be attached in advance.

The test plate 404 is provided in the vicinity of the washing machine, which is the object to be inspected 5. The surface of the test plate 404 is made of a material that does not reflect far infrared rays, and is, for example, a black aluminum plate whose surface is anodized, paper, a synthetic resin (plastic or the like) plate, and the like.

The test plate 404 is fixed so that its surface is vertically above or tilted at a predetermined angle from the horizontal direction. Further, a nozzle 403 is provided directly above the test plate 404. As a result, the fine water droplets 8 that naturally fall from the nozzle 403 reach the surface of the test plate 404. The test plate 404 and the nozzle 403 are provided at positions that do not interfere with the operation and movement of the far-infrared camera 50.

Since the fine water droplet 8 falling from the nozzle 403 assumes the residual water content in the object to be inspected 5 detected in the second inspection area 140, it falls from the nozzle 402 in the first inspection area 120. The size is preferably smaller than the water droplet 7.

Further, it is preferable that the distance between the nozzle 403 and the test plate 404 is not so long so that the fine water droplets 8 that have fallen on the test plate 404 do not scatter.

Next, the dehumidifying device 30 in the second inspection area 140 shown in FIG. 1 is a device that reduces the humidity of the space in which the object to be inspected 5 exists (humidity of the second inspection area 140). Further, the blower device 40 in the second inspection area 140 is a device that blows air toward the surface of the object to be inspected 5 moving in the second inspection area 140 and the surface of the internal piping or the like by the transport device 160. These dehumidifying devices 30 and blowing devices 40 are similar to the dehumidifying devices 30 and blowing devices 40 in the first inspection area 120, and one or more of them are installed.

In addition to the dehumidifying device 30 or the blower device 40, the inspected object 5 is used as a pretreatment device in various places in the inspection area 100 in order to further promote the drying of the surface of the inspected object 5 and the surface of the internal piping and the like. An air conditioner that raises the temperature of the space in which the air conditioner exists (the temperature of the inspection area 100), or a device that lowers the temperature of water leaked to the surface of the object 5 to be inspected and the surface of internal piping (for example, an air conditioner). A device for cooling the surface of the object to be inspected 5 and the surface of the internal piping or the like) may be installed (reference numeral 60).

The temperature measuring device 80 measures the temperature (air temperature) of the atmosphere in the second inspection area 140. The temperature measuring device 80 is installed to obtain a temperature difference from the temperature of the test plate 404 in which fine water droplets 8 are present on the surface.

The water leak inspection of the present embodiment is performed in each of the first inspection area 120 and the second inspection area 140 described above.
Specifically, the water leak inspection system 1 first photographs water droplets 7 on a trial basis by a visible light camera 20 installed in the first inspection area 120, and further, far infrared rays installed in the second inspection area 140. A far-infrared image of fine water droplets 8 is taken on a trial basis by the camera 50. The water leak inspection system 1 performs a preliminary inspection to confirm whether or not water droplets 7 and fine water droplets 8 are normally photographed on these captured images.

If there are no abnormalities in these preliminary inspections (successful), the water leak inspection system 1 acquires a visible light image of the inspected object 5 by the visible light camera 20, and further, the inspected object by the far infrared camera 50. The far-infrared image of 5 is acquired. Then, the water leak inspection system 1 performs a predetermined image determination process on these images to determine the presence or absence of water leakage in the inspected object 5.

Then, in the water leak inspection system 1 of the present embodiment, the portion of the inspected object 5 in which water leakage is erroneously detected (actually, water leaks) in each of the visible light image and the far infrared image of the inspected object 5. Predetermined mask data (visible light mask data, far-infrared mask data) are generated in advance in order to exclude the missing portion) from the target of the determination process, and these are used to obtain the visible light image of the object 5 to be inspected and the far-infrared light image. Performs image determination processing for infrared images.

Next, as shown in FIG. 1, an image processing device 200 is introduced in the water leak inspection system 1. The image processing device 200 is an information processing device (computer) installed in the inspection area 100 or another place (management center or the like). The image processing device 200 is communicably connected to the visible light camera 20, the far infrared camera 50, and the temperature measuring device 80 via a predetermined communication network 3. The communication network 3 is, for example, a wired or wireless communication network such as a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or a dedicated line.

Here, FIG. 4 is a diagram showing an example of the configuration of the image processing device 200. The image processing device 200 has a processor 20 such as a CPU (Central Processing Unit) as hardware.
1, a main storage device 202 such as RAM (Random Access Memory) and ROM (Read Only Memory), an auxiliary storage device 203 such as HDD (Hard Disk Drive) and SSD (Solid State Drive), and a keyboard, mouse, and touch panel. It includes an input device 204 including the above, an output device 205 including a monitor (display) and the like, and a communication device 206 for communicating with each device.

Further, the image processing device 200 has the functions of a camera control unit 210, a water droplet determination unit 220, a residual moisture determination unit 230, a mask area input unit 240, and a mask area output unit 250. The camera control unit 210 performs predetermined control on the visible light camera 20 and the far infrared camera 50. For example, the camera control unit 210 controls the articulated robot 55 to move the far-infrared camera 50 to a predetermined position or set the far-infrared camera 50 to a predetermined shooting direction. Further, the camera control unit 210 transmits a shooting instruction signal to the visible light camera 20 and the far infrared camera 50 at a predetermined plurality of timings (for example, a predetermined time interval or a predetermined time), and the visible light image and the far infrared ray. The image can be taken. Then, the camera control unit 210 acquires and stores the image taken by the visible light camera 20 or the far infrared camera 50 via the communication network 3. The articulated robot 55 may autonomously perform the above control, or the articulated robot 55 may also control the far-infrared camera 50.

The water drop determination unit 220 descends from the surface of the object to be inspected 5 or the surface of its internal piping, or falls from the object to be inspected 5 or its internal piping, based on a plurality of visible light images acquired by the camera control unit 210. Determine the presence or absence of water droplets. That is, the water droplet determination unit 220 includes a visible light image pre-inspection unit 229, a visible light mask region generation unit 221 and a visible light image determination unit 223.

From the visible light camera, the visible light mask region generation unit 221 captures a first visible light image of the surface of the structure having substantially the same structure as the object to be inspected and having no water leakage on the surface thereof. Based on each of the first visible light images acquired at a plurality of timings, the characteristic region of the visible light image in which the gradation difference extends in a predetermined direction with the progress of time is specified and specified. A visible light learning mode is executed in which a characteristic area and its surrounding area are set as a visible light mask area (visible light mask data).

The visible light image determination unit 223 acquires a second visible light image of the surface of the object to be inspected containing water at a plurality of timings, and corresponds to the visible light mask region from each acquired second visible light image. The characteristic of the visible light determination region, in which the regions excluding the portion to be specified are each specified as a visible light determination region, and the gradation difference extends in a predetermined direction with the progress of time based on each specified visible light determination region. When a specific region is specified, a visible light inspection mode for determining that a water leak has occurred in the object to be inspected containing the water is executed.

The visible light image pre-inspection unit 229 acquires visible light images of water droplets falling from a predetermined position from the visible light camera at a plurality of timings, and the gradation difference extends in a predetermined direction with the progress of time. When it is determined that the acquired visible light image region exists, the visible light image determination unit performs a water leak inspection of the object to be inspected.

Further, the water droplet determination unit 220 includes a visible light mask region addition unit 225. When the visible light image determination unit 223 erroneously detects a water leak, the visible light mask area addition unit 225 uses the visible light image used at that time as mask data.

That is, the visible light mask region addition unit 225 is said to be characteristic when water leakage does not occur in the region specified by the visible light image determination unit as a characteristic region of the visible light determination region. Area is set as a new visible light mask area.

Based on the far-infrared image acquired by the camera control unit 210, the residual water content determination unit 230 determines the presence or absence of water remaining on the surface of the object 5 to be inspected, which is determined to have no water droplets, or the surface of the internal piping thereof. .. That is, the residual moisture determination unit 230 includes a far-infrared image pre-inspection unit 239, a far-infrared mask region generation unit 231 and a far-infrared image determination unit 233.

From the far-infrared camera, the far-infrared mask region generation unit 231 captures a first far-infrared image of the surface of the structure having substantially the same structure as the object to be inspected and having no water leakage on the surface thereof. Based on each first far-infrared image acquired at multiple timings, the average temperature of the surface of the structure or the average temperature of the surface of the pipe filled with water is calculated, and the surface of the structure is calculated. A characteristic region in which a particularly low temperature is specified and the difference between the specified low temperature and the average temperature of the surface of the structure or the average temperature of the surface of a pipe filled with water is a predetermined value or more and the region thereof. Execute the far-infrared learning mode that sets the peripheral area as the far-infrared mask area (far-infrared mask data).

The far-infrared image determination unit 233 acquires a second far-infrared image of the surface of the object to be inspected containing water at a plurality of timings, and corresponds to the far-infrared mask region from each of the acquired second far-infrared images. Each region excluding the portion to be inspected is specified as a far-infrared determination region, and based on each identified far-infrared determination region, the surface of the object to be inspected corresponding to the far-infrared determination region is filled with the average temperature or water. The average temperature of the surface of the pipe to be inspected is calculated, the temperature of the surface of the corresponding object to be inspected is particularly low, and the specified low temperature is filled with the average temperature of the surface of the structure or water. A far-infrared inspection mode that determines that water leakage has occurred in the object to be inspected containing the water when a characteristic region where the difference from the average temperature of the pipe surface is specified is specified. Run.

The far-infrared image pre-inspection unit 239 acquires a far-infrared image of the surface of a predetermined member having a surface that does not reflect far-infrared rays and has water droplets arranged on the surface, calculates the average temperature of the surface, and calculates the average temperature of the surface. When a particularly low temperature among the surface temperatures is specified and a characteristic region where the difference between the specified low temperature and the average temperature is equal to or more than a predetermined value is specified, the object to be inspected by the far-infrared image determination unit is specified. Perform a water leak check.

Further, the residual moisture determination unit 230 includes a far-infrared mask region addition unit 235. When the far-infrared image determination unit 233 erroneously detects a water leak, the far-infrared mask region addition unit 235 uses the far-infrared image used at that time as mask data.

That is, if the far-infrared mask region addition unit 235 does not cause water leakage in the region specified by the far-infrared image determination unit as a characteristic region of the far-infrared determination region, the region is set. Set as a new far-infrared mask area.

Next, the mask area input unit 240 includes a visible light mask area input unit 241 and a far infrared mask area input unit 243.

The visible light mask area input unit 241 receives from the user an input for setting the visible light mask area for the object to be inspected for which water leakage does not occur on the surface thereof. The far-infrared mask area input unit 243 receives from the user an input for setting the far-infrared mask area for the object to be inspected for which water leakage does not occur on the surface thereof.

Next, the mask area output unit 250 includes a visible light mask area output unit 251 and a far-infrared mask area output unit 253.

The visible light mask area output unit 251 outputs the information of the set visible light mask area. The far-infrared mask area output unit 253 outputs the information of the set far-infrared mask area.

The function of the image processing device 200 as described above is performed by the hardware of the image processing device 200, or the processor 201 of the image processing device 200 reads out each program stored in the main storage device 202 or the auxiliary storage device 203. It is realized by executing.

Further, these programs are non-temporary readable by, for example, a secondary storage device, a non-volatile semiconductor memory, a hard disk drive, a storage device such as an SSD, or an information processing device such as an IC card, an SD card, or a DVD. Stored in a data storage medium.

<< Water leak inspection method >>
Next, the water leak inspection method performed by such a water leak inspection system 1 will be described.

<Menu screen>
FIG. 5 is a diagram showing an example of a menu screen on which the image processing device 200 is displayed when the image processing device 200 is started. On the menu screen 500, a pre-inspection mode button 517 that executes an inspection (pre-inspection mode) related to shooting of visible light images and far-infrared images, which is performed before the water leak inspection for the object 5 to be inspected, and input from the user. The inspection mode button 511 that executes the inspection (inspection mode) of each inspected object 5 by the image determination process when the image is accepted, and the mask data generation of the inspected object 5 when the input from the user is accepted (inspection mode). The learning mode button 515 that executes the learning mode) and the automatic inspection mode button 513 that automatically executes the pre-inspection mode, the learning mode, and the inspection mode continuously when the input from the user is received are displayed. ..
Hereinafter, the processing (automatic inspection processing) of the water leak inspection system 1 performed when the automatic inspection mode button 513 is pressed will be described.

<Automatic inspection process>
FIG. 6 is a flow chart illustrating an example of the automatic inspection process. First, the water leak inspection system 1 executes the pre-inspection mode (s1, s2). That is, the water leak inspection system 1 executes a preliminary inspection (visible light image preliminary inspection process) regarding the visible light image by taking a picture of the water droplet dropped by the first dropping system 400 by the visible light camera 20 (s1). .. Further, the water leak inspection system 1 executes a pre-infrared image (far-infrared image pre-inspection process) by taking a picture of the water droplet dropped by the second drip system 450 by the far-infrared camera 50 (s2). .. The details of the s1 process (visible light image pre-inspection process) and the s2 process (far-infrared image pre-inspection process) will be described later.

When it is determined that there is no abnormality in the visible light image pre-inspection process and the far-infrared image pre-inspection process, the water leak inspection system 1 executes a learning mode for generating mask data (s3, s4). That is, the water leak inspection system 1 has a water leak having a structure substantially the same as that of the inspected object 5 (manufactured separately from the inspected object 5 and manufactured separately from the inspected object 5) in which it is confirmed that no water leakage occurs. A visible light image (which may be a test product or the like for which it has been confirmed that the above does not occur) is taken in the first inspection area 120, and visible light mask data is generated by this (s3). Further, the water leak inspection system 1 captures a far-infrared image of the object 5 to be inspected in the second inspection area 140, thereby generating far-infrared mask data (s4). The details of the s3 process (visible light learning process) and the s4 process (far infrared ray learning process) will be described later.

Next, the water leak inspection system 1 executes an inspection mode for executing a water leak inspection of the inspected object 5 (the inspected object 5 that may cause water leakage) to be inspected (s5, s7).

That is, the water leak inspection system 1 captures a visible light image of the object 5 to be inspected in the first inspection area 120, and masks the captured visible light image with visible light mask data generated in the learning mode. Execute the process. Then, the water leak inspection system 1 performs image determination processing on the masked visible light image to determine whether or not there is a water leak in the object to be inspected 5 to be inspected (for example, whether or not water droplets are present). Is determined (s5).

Further, the water leak inspection system 1 captures a far-infrared image of the object 5 to be inspected in the second inspection area 140, and masks the captured far-infrared image with the far-infrared mask data generated in the learning mode. Execute the process. Then, the water leak inspection system 1 performs image determination processing on the masked far-infrared image to determine whether or not there is water leakage in the object 5 to be inspected (for example, fine residual water is present). Whether or not to do so) is determined (s7).

The details of the s5 process (visible light image determination process) and the s7 process (far infrared image determination process) will be described later.

When the above processing is completed, the water leak inspection system 1 starts from s3 in order to execute the learning mode again for the product when the water leak inspection is performed on the product (object 5 to be inspected) different from the above. (S9: YES). On the other hand, if a water leak inspection is not performed on a different type of product (object 5 to be inspected), the automatic inspection process ends (s9: NO, s10).

Next, the details of each process in the automatic inspection process will be described.
<Preliminary inspection mode: Visible light image pre-inspection processing>
FIG. 7 is a flow chart illustrating an example of the visible light image pre-inspection process. First, in the first inspection area 120, the water supply device 401 is driven to supply water to the nozzle 402, from which the water droplet 7 falls (s301).

Each visible light camera 20 provided in the first inspection area 120 takes a visible light image of the falling water droplet 7 at a predetermined time interval (s303). Specifically, for example, each visible light camera 20 shoots a visible light image at a shooting interval of 30 to 60 frames / sec. Then, each visible light camera 20 transmits each captured image (visible light image) to the image processing device 200 (s305).

When the image processing device 200 receives each visible light image (s307), the image processing apparatus 200 determines the presence or absence of falling water droplets 7 (water droplets falling from the nozzle 403) based on each received visible light image (s307).
Preliminary water droplet determination process) is performed (s309). The details of the pre-water droplet determination process will be described later.

If it is determined by the pre-water droplet determination process that there are water droplets to fall (s311: YES), the image processing apparatus 200 ends the process without issuing a warning (s312).

On the other hand, when it is determined that there are no falling water droplets 7 (s311: NO), the image processing device 200 issues a warning (for example, displays a predetermined screen indicating that there are no water droplets 7) and emits an alarm sound. Etc.) (s313).

In the first inspection area 120, when the above warning is issued within a predetermined time (determined as having an abnormality) (s315: YES), this process ends (s321). In this case, the inspector or the like implements measures such as repair and re-inspects (s317). For example, an inspector or the like confirms whether or not there is a defect in the visible light camera 20, confirms the first dropping system 400, and the like.

The warning may not be issued by the image processing device 200, but may be issued by a warning device other than the image processing device 200 (for example, a predetermined lighting device or an alarm device). Further, the image processing device 200 may transmit a predetermined warning signal to the visible light camera 20 via the communication network 3, and the visible light camera 20 may issue the warning.

On the other hand, in the first inspection area 120, when the above warning is not issued within a predetermined time (determined as no abnormality) (s315: NO), the image processing apparatus 200 presents that it has passed (s319) and processes. Is terminated (s321). That is, the image processing device 200 issues a message indicating that the preliminary inspection of the visible light image has been successful, and ends the processing (for example, displaying a predetermined screen indicating acceptance, issuing an alarm sound, etc.).
Here, the details of the pre-water droplet determination process will be described.

(Preliminary water drop judgment processing)
FIG. 8 is a flow chart illustrating the details of the pre-water droplet determination process. First, the image processing device 200 creates information on the gradation difference between the visible light images based on the received visible light images (s351). Specifically, for example, the image processing apparatus 200 calculates the difference in gradation (gradation difference) between the image at a certain time and the image after 1/30 to 1/60 seconds at the same position.

Then, the image processing apparatus 200 specifies a region having a large gradation difference as an existing region of the water droplet 7 based on the gradation difference information generated in s351 (s353). Specifically, for example, the image processing device 200 specifies all points where the gradation difference is larger than a predetermined threshold value for each visible light image.

The image processing device 200 is a region having a large gradation difference specified by s353 (a region where water droplets 7 exist).
Determines whether or not is stretched downward in a predetermined direction with the progress of time (s355). Specifically, for example, the image processing apparatus 200 specifies the time series (for example, the shooting order) of each visible light image, and then the set of the points in each visible light image is continuous vertically downward. It is a region (set) of points, and it is determined whether or not the length of this region (for example, the number of consecutive points) increases with time to reach a predetermined length (for example, a predetermined number). do. The predetermined length is set according to, for example, the shooting interval (number of frames) of the visible light camera 20.

In this way, when it is determined that there is a region having a large gradation difference that extends downward in a predetermined direction with the progress of time (s355: YES), the image processing apparatus 200 drops water droplets 7. It is determined that there is (s357), and the pre-water droplet determination process is completed (no abnormality) (s359). On the other hand, when it is determined that there is no region having a large gradation difference downward extending in a predetermined direction with the progress of time (s355: NO), the image processing apparatus 200 determines that the water droplet 7 has not fallen. The determination (abnormality) (s361) is made, and the pre-water droplet determination process is completed (s359).

By the above visible light image pre-inspection process, the first dropping system 400 and the visible light camera 20 detect the water droplet 7 falling from the nozzle 402, so that the abnormality of the water leak inspection system 1 (visible light camera 20 etc.) is detected. It can be detected early. As a result, by repairing the abnormal part at an early stage, it is possible to eliminate the detection omission and prevent the detection of water droplets from being overlooked.

<Pre-inspection mode: Far-infrared image pre-inspection processing>
FIG. 9 is a flow chart illustrating an example of the far-infrared image pre-inspection process. First, the water supply device 405 in the second inspection area 140 is driven to supply water to the nozzle 403 (s401), from which fine water droplets 8 fall and fall on the surface of the test plate 404.

At this time, the dehumidifying device 30 and the ventilation device 40 installed in the second inspection area 140 are operating, and these reduce the humidity of the second inspection area 140 and blow air to the test plate 404. It shall be.

The far-infrared camera 50 attached to the articulated robot 55 photographs the surface of the test plate 404 in which fine water droplets 8 are arranged on the surface thereof, and acquires the image (far-infrared image) (s403).

The far-infrared camera 50 transmits the captured image (far-infrared image) to the image processing device 200 (s405).

When the image processing apparatus 200 receives the far-infrared image (s407), the image processing apparatus 200 performs a process (pre-residual moisture determination process) of determining whether or not moisture is present on the surface of the test plate 404 based on the received far-infrared image. Execute (s409). The details of the pre-residual moisture determination process will be described later.

When it is determined by the pre-residual water content determination process that the fine water droplet 8 does not exist (s411: NO), the image processing apparatus 200 issues a warning that the fine water droplet 8 could not be detected (for example,). A predetermined screen display indicating that there is no residual water, an alarm sound, etc.) (s413).

On the other hand, when it is determined by the pre-residual water content determination process that fine water droplets 8 are present (s411: YES), the image processing device 200 determines the temperature of the atmosphere of the second inspection area 140 measured by the temperature measuring device 80. And, it is determined whether or not the temperature difference from the average temperature of the surface of the far infrared image is within a predetermined range (s415).

When the temperature difference is within a predetermined range (s415: YES), the image processing apparatus 200 issues a warning to that effect (for example, displays a predetermined screen and emits an alarm sound) (s417). On the other hand, if the temperature difference is not within the predetermined range (s415: NO), this process ends (s419).

When any of the above warnings is issued within a predetermined time (determined as having an abnormality) (s413, s417, s421: YES), this process ends (s427). Then, in this case, the inspector or the like in the second inspection area 140 repairs the abnormal portion (s423). For example, an inspector or the like confirms whether or not there is a defect in the far-infrared camera 50, confirms the second dropping system 450, and the like.

On the other hand, if any of the above warnings is not issued within the predetermined time (determined as no abnormality) (s413, s417, s421: NO), the pre-inspection of the far-infrared image is successful (no abnormality). (S425), the process is terminated (s427). That is, the image processing device 200 issues a message indicating that the pre-inspection of the far-infrared image has been successful (for example, displaying a predetermined screen indicating acceptance, issuing an alarm sound, etc.).

Note that each of the above warnings may not be issued by the image processing device 200, but may be issued by a predetermined warning device (for example, a predetermined lighting device or alarm device) other than the image processing device 200. Further, the image processing device 200 may transmit a predetermined warning signal to the far-infrared camera 50 via the communication network 3, and the far-infrared camera 50 may issue the warning.
Here, the details of the pre-residual moisture determination process will be described.

(Pre-residual moisture determination process)
FIG. 10 is a flow chart illustrating the details of the pre-residual moisture determination process. First, the image processing apparatus 200 calculates the temperature distribution on the surface of the test plate 404 recorded in the far-infrared image based on the received far-infrared image (converts the image into temperature; s451).

Then, the image processing apparatus 200 identifies a region (determination region) where the temperature is particularly low and the temperature thereof based on the calculated temperature distribution, and calculates the average temperature of the surface of the test plate 404 recorded in the far-infrared image. (S453).

By promoting vaporization using the blower 40, the present inventors can easily identify the above-mentioned determination region (particularly the region where the temperature is low), and greatly improve the detection accuracy of the residual moisture by the far-infrared image. I came up with the idea of being able to do it.

The image processing apparatus 200 calculates the difference between the temperature specified by s453 and the average temperature of the surface of the test plate 404 (s455). When this difference is equal to or greater than a predetermined threshold value (s455: YES), the image processing apparatus 200 determines that fine water droplets 8 are present in the determination region (s457), and the pre-residual moisture determination process ends (abnormality). None) (s461). On the other hand, when the difference between the average temperature and the low temperature is less than a predetermined threshold value (s455: NO), the image processing apparatus 200 determines that the fine water droplet 8 does not exist in the determination region (s459). , The pre-residual moisture determination process is completed (with abnormality) (s461).

By the above far-infrared image pre-inspection process, the second dropping system 450 and the far-infrared camera 50 detect the fine water droplets 8 falling from the nozzle 403, so that the automatic inspection system 1 (far-infrared camera 50, etc.) is abnormal. Can be detected early. As a result, by repairing the abnormal portion at an early stage, it is possible to eliminate the detection omission and prevent the detection of the fine water droplet 8 (residual water content) from being overlooked.
Next, the processing in the learning mode will be described.

<Learning mode: Visible light learning process>
FIG. 11 is a flow chart illustrating an example of visible light learning processing. First, in the first inspection area 120, water is supplied to the inspected object 5 (which does not cause water leakage), which is the target of the generation of the water leakage inspection mask, and the operation (operation) of the inspected object 5 is performed. Start (s11). Specifically, for example, a predetermined amount of water is injected into the water storage portion (drum or the like) of the washing machine, which is the object to be inspected 5, and the power is turned on to start the washing operation. This step may be performed automatically by a predetermined water supply device, or may be performed by an inspector or the like.

When the water-supplied object 5 to be inspected is carried into the entrance 122 of the first inspection area 120, the conveyor 160 in the first inspection area 120 starts to carry the inspected object 5, and the visible light camera 20 takes a picture. The inspected object 5 is transported to the area (s13).

At this time, it is assumed that the dehumidifying device 30 and the ventilation device 40 in the first inspection area 120 are operating, and these reduce the humidity of the first inspection area 120 and blow air to the object to be inspected 5. ..

When each visible light camera 20 provided in the first inspection area 120 detects that the object to be inspected 5 has entered its own imaging range, it photographs the object to be inspected 5 at predetermined time intervals (s15). ). Specifically, for example, each visible light camera 20 captures a visible light image (first visible light image) of the object 5 to be inspected at an imaging interval of 30 to 60 frames / sec. Then, each visible light camera 20 transmits each captured visible light image (frame) to the image processing device 200 (s17).

When the image processing apparatus 200 receives each visible light image (s19), the image processing apparatus 200 generates visible light mask data of the object to be inspected 5 based on each received visible light image (s21). The details of this visible light mask data generation process will be described later.

When the visible light mask data generation process is completed, the object 5 to be inspected is transported from the outlet 124 of the first inspection area 120 to the second inspection area 140 by the transport device 160 (s23). As a result, the visible light learning process for one inspected object 5 is completed.

At this time, if the number of objects 5 to be inspected for which the visible light mask data generation process has been performed reaches the specified number (s25: YES), the visible light learning process ends (s27), and the visible light mask data generation process is performed. If the number of the inspected objects 5 performed has not reached the specified number (s25: NO), the process of s11 is repeated in order to perform the visible light learning process on the remaining inspected object 5.

Here, the details of the visible light mask data generation process will be described.
(Visible light mask data generation process)
FIG. 12 is a flow chart illustrating the details of the visible light mask data generation process. First, the image processing device 200 creates information on the gradation difference regarding the visible light image received from the image processing device 200 (s51). Specifically, for example, the image processing apparatus 200 calculates a gradation difference (gradation difference) between frames for each point in the visible light image of the object 5 to be inspected. Specifically, for example, the image processing device 200 displays the gradation of each point of the visible light image (frame) at a certain time and the visibility after a predetermined time (for example, 1/30 to 1/60 second). The difference (gradation difference) from the gradation of each corresponding point of the optical image (frame) is calculated.

Then, the image processing apparatus 200 identifies a characteristic region having a large gradation difference in the visible light image region based on the gradation difference information generated in s51 (s53). Specifically, for example, the image processing apparatus 200 specifies all points where the gradation difference is larger than a predetermined threshold value.

As an example of a characteristic region having a large gradation difference, a region in which water droplets are present can be generally considered. However, it has already been confirmed that water leakage does not occur in the object 5 to be inspected in the visible light mask data generation process. Therefore, the characteristic region where the gradation difference is large here is, for example, from the portion of the surface of the object to be inspected 5 that has been subjected to a predetermined gloss treatment, or the object to be inspected 5 or its internal piping. It is a part that may be falsely detected as water, such as exposed metal parts (screws, metal clips, etc.).

It is preferable that the threshold value is set lower than that in the pre-inspection mode and the inspection mode (that is, the mask is set to be slightly wider).

The image processing apparatus 200 determines whether or not the characteristic region having a large gradation difference specified in s53 extends downward in a predetermined direction with the progress of time (s55). That is, the image processing device 200 determines whether or not the behavior is similar to that of water droplets, and specifically, for example, after specifying the time series of each frame of the visible light image, the above-mentioned in each frame. Each set of points is a region (set) of points that are continuous vertically downward, and the length of this region (for example, the number of consecutive points) increases with time to a predetermined length (for example, for example). It is determined whether or not the predetermined number) has been reached. The predetermined length is set according to, for example, the shooting interval (number of frames) of the visible light camera 20.

As described above, when it is determined that there is a region having a large gradation difference that extends downward in a predetermined direction with the progress of time (s55: YES), the image processing apparatus 200 determines this region and the region. The peripheral area in the vicinity is determined to be the visible light mask area of the object 5 to be inspected, and the data representing this visible light mask area (visible light mask data. For example, the value of the mask portion is 0 and the value of the non-mask portion is 1. A certain data corresponding to a visible light image) is generated and stored (s57).

The image processing apparatus 200 confirms whether or not there is another unprocessed visible light image (s59), and if there is another unprocessed visible light image, repeats the processing from s51 for the visible light image. (S59: NO), if there is no other unprocessed visible light image (s59: YES), the visible light mask data generation process ends (s61).

By generating the visible light mask data corresponding to the visible light image of the object 5 to be inspected in this way, it is possible to prevent erroneous detection of water droplets.

Next, the far-infrared learning process in the learning mode will be described.
<Learning mode: Far infrared learning processing>
FIG. 13 is a flow chart illustrating an example of far-infrared learning processing. First, the conveyor 160 causes the object 5 to be inspected (which does not cause water leakage) sent from the first inspection area 120 to enter the entrance 142 of the second inspection area 140 (s71). At this time, the dehumidifying device 30 and the ventilation device 40 installed in the second inspection area 140 are operated, and these reduce the humidity of the second inspection area 140 and blow air to the inspected object 5. It is assumed that there is.

When the inspected object 5 that has entered the second inspection area 140 stops at a predetermined position, the far-infrared camera 50 installed at the tip of the articulated robot 55 captures an image of each region on the surface of the inspected object 5 (first). Far-infrared image) is taken (s73).

The far-infrared camera 50 transmits each captured far-infrared image to the image processing device 200 (s75).

When the image processing device 200 receives each far-infrared image (s77), the image processing apparatus 200 generates far-infrared mask data based on each received far-infrared image (s79). The details of this far-infrared mask data generation process will be described later.

When the far-infrared mask data generation process is completed, the object 5 to be inspected is transported to the exit of the second inspection area 140 by the transport device 160 (s81). As a result, the far-infrared learning process for one inspected object 5 is completed.

At this time, if the number of objects 5 to be inspected for which the far-infrared mask data generation processing has been performed reaches the specified number (s83: YES), the far-infrared learning processing is completed (s85), and the far-infrared mask data generation processing is performed. If the number of objects to be inspected 5 has not reached the specified number (s83: NO)
, The process of s71 is repeated in order to perform the far-infrared learning process on the remaining object 5 (a different type of product that does not cause water leakage).

Here, the details of the far-infrared mask data generation process will be described.
(Far infrared mask data generation processing)
FIG. 14 is a flow chart illustrating the details of the far-infrared mask data generation process. First, the image processing device 200 calculates the temperature distribution on the surface of the object to be inspected 5 from the received far-infrared image of the object to be inspected 5 (converts the image into temperature; s91).

Then, the image processing apparatus 200 identifies a characteristic region of the inspected object 5 having a particularly low temperature and the temperature thereof based on the calculated temperature distribution, and fills the surface of the inspected object 5 with the average temperature or water. The average temperature of the surface of the pipe (the pipe in the object 5 to be inspected) is calculated (s93). The region where the temperature is particularly low in this case may be, for example, a region having a temperature lower than a predetermined temperature (the temperature of the region in that case may be, for example, the average temperature of the surface of the object 5 to be inspected or water. It may be a region where the temperature is relatively low (referred to as the average temperature of the filled pipe surface). Further, the predetermined temperature is set according to, for example, the specific heat of the material (metal, synthetic resin, rubber, etc.) of the object 5 to be inspected.

An example of a characteristic region where the temperature is particularly low is generally a region where water droplets are present, but it has already been established that water leakage does not occur in the inspected object 5 in the far-infrared mask data generation process. It has been confirmed. Therefore, the region where the temperature is particularly low here may be erroneously detected as water, for example, a metal part (screw, metal clip, etc.) exposed on the surface of the object 5 to be inspected or its internal piping. It's a part.

It is preferable that the predetermined temperature is set lower than that in the pre-inspection mode or the inspection mode (that is, the mask is set to be slightly wider).

The image processing apparatus 200 calculates the difference between the temperature specified in s93 and the average temperature of the surface of the object 5 to be inspected or the average temperature of the surface of the pipe filled with water (s95). When this difference is equal to or greater than a predetermined threshold value (s95: YES), the image processing apparatus 200 identifies a region of the temperature specified in s93 (s96), and this region and the peripheral region (far) in the vicinity of the region. Data representing the infrared mask region) (far-infrared mask data, for example, matrix data corresponding to a far-infrared image in which the value of the mask portion is 0 and the value of the non-mask portion is 1,) is stored (s98).

The image processing device 200 confirms whether or not there is another unprocessed far-infrared image (s99), and if there is another unprocessed far-infrared image, repeats the processing from s91 for the far-infrared image. (S99: YES), if there is no other unprocessed far-infrared image (s99: NO), the far-infrared mask data generation process ends (s101).

By generating the far-infrared mask data corresponding to the far-infrared image of the object 5 to be inspected in this way, it is possible to prevent erroneous detection of fine residual moisture.

By the above learning mode, the water leak inspection system 1 generates visible light mask data and far infrared mask data for the object to be inspected 5.

Next, the water leak inspection system 1 uses these mask data to perform a water leak inspection of the object 5 to be inspected for water leakage (inspection mode). First, the visible light image determination process in the inspection mode will be described.

<Inspection mode: Visible light image judgment processing>
FIG. 15 is a flow chart illustrating an example of visible light image determination processing. First, water is supplied to the inspected object 5 (the inspected object 5 that may have leaked water) to be inspected for water leakage, and the operation (operation) of the inspected object 5 is started (s111). ). This point is the same as s11 of the visible light learning process.

The subsequent processing of s113, s115, and s117 (capturing and transmitting a visible light image) is the same as that of the visible light learning processing s13, s15, and s17.

Then, when the image processing device 200 receives a visible light image (second visible light image) composed of a plurality of frames from the visible light camera 20 (s119), each received image and the visible light mask data generated in the learning mode are obtained. Based on the above, a process for determining the presence or absence of water droplets (water droplets due to water leakage of the inspected object 5) that descends on the surface of the inspected object 5 and the surface of its internal piping or the like or falls from the inspected object 5 (visible light image). Mask processing) is performed (s121). The details of the visible light image mask processing will be described later.

When it is determined by the visible light image mask processing that there are water droplets falling or falling (s123: YES), the image processing apparatus 200 issues a warning (for example, a predetermined screen display indicating that there are water droplets). Perform, emit an alarm sound, etc.) (s125). If it is determined that there are no falling or falling water droplets (s123: NO), the image processing apparatus 200 ends the processing without issuing a warning (s127).

When the above warning is issued (s129: YES), in the first inspection area 120, the inspector or the like determines that the inspected object 5 is a defective product, and the inspected object 5 is in the first inspection area 120. By the time it reaches the outlet 124, the operation of the conveyor 160 is stopped to separate the object 5 to be inspected from the conveyor 160 (s131). In addition, this work may be performed automatically by a predetermined conveyor or the like. As a result, the water leakage inspection in the first inspection area 120 for the inspected object 5 is completed (s133).

On the other hand, if the above warning is not issued (s129: NO), the object 5 to be inspected has passed the water leakage inspection in the first inspection area 120 (s135). The object 5 to be inspected is moved from the outlet 124 of the first inspection area 120 to the second inspection area 140 by the transport device 160.

The warning may not be issued by the image processing device 200, but may be issued by a predetermined warning device other than the image processing device 200 (for example, a predetermined lighting device or an alarm device). Further, the image processing device 200 may transmit a predetermined warning signal to the visible light camera 20 via the communication network 3, and the visible light camera 20 may issue the warning.
Here, the details of the visible light image mask processing will be described.

(Visible light image mask processing)
FIG. 16 is a flow chart illustrating the details of the visible light image mask processing. First, the image processing device 200 performs mask processing on each visible light image (frame) of the object 5 to be inspected received from the visible light camera 20 using visible light mask data (s150).

The image processing apparatus 200 creates information on the gradation difference of the visible light image (frame) masked by s150 (s151). Specifically, for example, the image processing apparatus 200 has a gradation of each point in a visible light image (frame) at a certain time, and after a predetermined time (for example, 1/30 to 1/60 second). The difference (gradation difference) from the gradation of each corresponding point of the visible light image (frame) is calculated.

Then, the image processing apparatus 200 identifies a region of the visible light image having a large gradation difference as a region where water droplets exist (s153), based on the gradation difference information generated in s151. Specifically, for example, the image processing device 200 specifies all points where the gradation difference is larger than a predetermined threshold value for each visible light image.

Similar to s55 in the visible light learning mode, the image processing device 200 determines whether or not the region having a large gradation difference (the region where water droplets exist) specified in s153 extends downward in a predetermined direction with the progress of time. (S155).

When it is determined that there is a characteristic region having a large gradation difference, which extends downward in a predetermined direction with the progress of time (s155: YES), the image processing apparatus 200 has water droplets on the object to be inspected. It is determined that the surface of No. 5 or the surface of the internal piping thereof or the like is descended, or water droplets are dropped from the object to be inspected 5 (s157), and the visible light image mask processing is completed (s159). On the other hand, when it is determined that there is no characteristic region having a large gradation difference, which extends downward in a predetermined direction with the progress of time (s155: NO), the image processing apparatus 200 is covered with water droplets. It is determined that neither the surface of the inspection object 5 nor the surface of the internal piping or the like has fallen and has not fallen from the inspection object 5 (s161), and the visible light image mask processing is completed (s159). ..

In s155, the image processing device 200 determines whether or not the shape of the specified region is a predetermined shape (for example, a circular shape), and determines that the shape of the specified region is the predetermined shape. The specified area may be the determination target only in. By additionally determining the shape in this way, water leakage can be detected more accurately.

In this way, by masking the visible light image taken by the visible light camera 20 and then detecting the falling or falling water droplets, water leakage from the inspected object 5 and the inside thereof can be detected without error. can do.

Subsequently, the far-infrared image determination process in the inspection mode will be described.
<Inspection mode: Far infrared image judgment processing>
FIG. 17 is a flow chart illustrating an example of far-infrared image determination processing. First, the conveyor 160 causes the inspected object 5 (which may cause water leakage) sent from the first inspection area 120 to enter the entrance 142 of the second inspection area 140 (s171). This point is the same as s71 of the far-infrared learning process.

The subsequent processing of s173 and s175 (capturing and transmitting a far-infrared image) is the same as that of the far-infrared learning processing s73 and s75.

Subsequently, when the image processing device 200 receives the far-infrared image (second far-infrared image) from the far-infrared camera 50 (s177), the surface of the object 5 to be inspected and its internal piping and the like are based on the received far-infrared image. Moisture on the surface of the surface (a small amount of water droplets that could not be detected by the visible light camera 20 and
A process (far-infrared image mask process) for determining whether or not a descent or a drop (water droplets or the like whose movement speed is slow) remains is executed (s179). Details of the far-infrared image mask processing will be described later.

When it is determined by the far-infrared image mask processing that there is residual moisture (s181: YES), the image processing apparatus 200 issues a warning (for example, displays a predetermined screen indicating that there is residual moisture). It emits an alarm sound, etc.) (s183). On the other hand, when it is determined that there is no residual water (s181: NO), the image processing apparatus 200 ends the processing without issuing a warning (s185).

When the above warning is issued (s187: YES), in the second inspection area 140, the inspector or the like temporarily operates the transport device 160 until the inspected object 5 reaches the exit 144 of the second inspection area 140. The object to be inspected 5 is detached from the transport device 160 by, for example, stopping the object (s189). In addition, this work may be performed automatically by a predetermined conveyor or the like.

On the other hand, if the above warning is not issued (s187: NO), it is assumed that the inspected object 5 has passed the water leakage inspection, and the inspected object 5 moves to the exit 144 of the second inspection area 140. , The water leakage inspection for the inspected object 5 is completed (s190).

The warning may not be issued by the image processing device 200, but may be issued by a predetermined warning device other than the image processing device 200 (for example, a predetermined lighting device or an alarm device). Further, the image processing device 200 may transmit a predetermined warning signal to the far-infrared camera 50 via the communication network 3, and the far-infrared camera 50 may issue the warning.

Here, the details of the far-infrared image mask processing will be described.
(Far infrared image mask processing)
FIG. 18 is a flow chart illustrating the details of the far-infrared image mask processing. First, the image processing device 200 performs mask processing using far-infrared mask data on each far-infrared image received from the far-infrared camera 50 (s190).

The image processing apparatus 200 calculates the temperature distribution on the surface of the object to be inspected 5 from the far-infrared image masked by s190 (converts the image into temperature; s191).

Then, based on the calculated temperature distribution, the image processing apparatus 200 identifies the surface regions of the object 5 to be inspected having a particularly low temperature, the surface regions of the internal piping and the like, and the temperatures thereof, and records them in a far-infrared image. The average temperature of the surface of the object to be inspected 5 or the surface of the pipe filled with water (internal piping in the object 5 to be inspected, etc.) is calculated (s193). The region in this case may be, for example, a place showing the minimum temperature or a region having a temperature lower than a predetermined temperature (the temperature of the region in that case may be, for example, the object to be inspected). The average temperature of the surface of 5 or the average temperature of the pipe surface filled with water). Further, the predetermined temperature here is set according to, for example, the specific heat of the material (metal, synthetic resin, rubber, etc.) of the object 5 to be inspected.

By promoting vaporization using the blower 40, the present inventors can easily identify the above-mentioned determination region (particularly the region where the temperature is low), and greatly improve the detection accuracy of the residual moisture by the far-infrared image. I got the finding that it can be done.

The image processing apparatus 200 calculates the difference between the particularly low temperature specified in s193 and the average temperature of the surface of the object 5 to be inspected or the average temperature of the surface of the pipe filled with water (s195). When this difference is equal to or greater than a predetermined threshold value (s195: YES), the image processing apparatus 200 determines that the residual water is present in the region specified by s193 (s197), and the far-infrared image mask processing ends (s). s199). On the other hand, when the difference between the average temperature of the surface of the object 5 to be inspected 5 or the average temperature of the pipe surface filled with water and the low temperature is less than a predetermined threshold value (s195: NO), image processing is performed. The apparatus 200 determines that the residual water does not exist in the region specified by s193 (s201), and the far-infrared image masking process ends (s199). When a plurality of regions (low temperature) are specified in s193, the processing of s195 may be performed for each of the regions (low temperature).

In s195, the image processing device 200 determines whether or not the shape of the specified region is a predetermined shape (for example, a circular shape), and determines that the shape of the specified region is the predetermined shape. The specified area may be the determination target only in. By additionally determining the shape in this way, the residual moisture can be detected more accurately.

In this way, by masking the far-infrared image taken by the far-infrared camera 50 to detect fine residual water, the residual water in the inspected object 5 and the inside thereof can be found early and without error. be able to.

<Screen examples other than the menu screen>
Next, screen examples other than the menu screen 500 described above will be described.

(Learning mode screen)
FIG. 19 is a diagram showing an example of a learning mode screen displayed when the learning mode button 515 is pressed on the menu screen 500. The learning mode screen 600 has an automatic generation start button 611 for automatically executing the learning mode to generate visible light mask data and far-infrared mask data, and the captured visible light image and far-infrared image. A manual creation button 613 for the user to manually set the visible light mask area and the far-infrared mask area and manually create the visible light data and the far-infrared mask data, and an end button 615 for ending the learning mode are displayed. Will be done.

(Screen for creating visible light mask data)
FIG. 20 is an example of a visible light mask data creating screen displayed on the image processing apparatus 200 when the visible light mask data is automatically created as a result of pressing the automatic generation start button 611 on the learning mode screen 600. It is a figure which shows. On the visible light mask data creation screen 620, the state of each visible light mask area 301-304 automatically generated for the captured visible light image is displayed, and then the visible light mask data corresponding to these is displayed. Is created automatically.

(Screen for creating far-infrared mask data)
FIG. 21 is a diagram showing an example of a far-infrared mask data creating screen displayed on the image processing apparatus 200 when the automatic generation start button 611 is pressed and far-infrared mask data is automatically created. On the far-infrared mask data creation screen 640, the state of each far-infrared mask area 310-313 automatically generated for the captured far-infrared image is displayed, and then the corresponding far-infrared mask data is displayed. Is created automatically.

(Water leak inspection result display screen)
FIG. 22 is a diagram showing an example of a water leak inspection result display screen showing the result of the automatic inspection process. On the water leak inspection result display screen 800, the first display column 802 for displaying the result of the inspection by the visible light image performed in the first inspection area 120 and the inspection by the far infrared image performed in the second inspection area 140 are displayed. A second display field 804 for displaying the result of the above and a mask addition button 806 described later are provided. In the first display column 802 and the second display column 804, if a water leak is detected as a result of the inspection, "water leak" is displayed, and if no water leak is detected, "pass" is displayed.

Here, even though "water leakage" is displayed in any of the inspections displayed by the first display column 802 and the second display column 804, water leakage is subsequently confirmed by visual inspection by an inspector or the like. If it is found that no water leak has occurred (passed), the inspection has falsely detected a water leak.

In such a case, the user specifies the display field 805 for the inspection in which water leakage is erroneously detected, and then presses the mask addition button 806, so that the image processing apparatus 200 is used in the inspection in which water leakage is erroneously detected. The image (visible light image or far infrared image) before mask processing is converted into new mask data and stored.

In the example of FIG. 22, the inspected object 5 in the visible light inspection of the first inspection area 120 is “1-2” and the inspected object 5 in the far infrared inspection of the second inspection area 140 is “2-1”. When the corresponding mask addition button 806 is pressed, the image processing apparatus 200 sets new visible light mask data and far infrared rays based on the visible light image and the far infrared image before the mask processing. Generates and stores mask data.

In this way, it is possible to reduce erroneous detection by adding and learning the image of the portion erroneously detected as water leakage (actually no water leakage) as new mask data.

As described above, in the method for generating a mask for water leakage inspection of the present embodiment, first, (1) the gradation difference is large from the visible light image of the inspected object in which water does not exist on the surface and the surface of the internal piping. The characteristic region in which the region extends in a predetermined direction (for example, downward) with the progress of time is identified, and the visible light mask region is set to include the characteristic region. After that, a visible light image of the object to be inspected is acquired, and a characteristic area is specified for the area excluding the visible light mask area from this visible light image, so that water leakage occurs. Is determined. (2) The far-infrared image also has a characteristic region, that is, a particularly low temperature among the temperatures of the surface of the object to be inspected, the average temperature of the surface of the object to be inspected, or the average of the surface of the pipe filled with water. Identify the region where the difference from the temperature is large, and set the far-infrared mask region to include this region. After that, a far-infrared image of the object to be inspected is acquired, and a characteristic region is specified for the region excluding the far-infrared mask region from this far-infrared image, so that moisture remains. judge.

In this way, mask data is generated for each of the visible light image and the far infrared image of the inspected object in which water does not exist, and there is a possibility that water leakage has occurred using these mask data. By performing image determination of the object to be inspected, it is possible to prevent erroneous detection of water leakage, such as determining that there is water leakage even though there is no actual water leakage. As described above, according to the water leak inspection mask generation method of the present embodiment, water leakage from the inspected object can be accurately detected.

[Second Embodiment]
In the actual operation of the water leak inspection method as shown in the first embodiment, various inspection errors may occur. For example, even if the inspected object 5 is a product of the same type, a design error (dimensions of the housing and various parts, etc.) occurs for each individual. Further, even when the inspected object 5 is transported by the transporting device 160, an error occurs in the fixing position of the inspected object 5 to the transporting device 160 for each individual. If such an error is not taken into consideration, an error may occur in the captured image, resulting in erroneous detection. Therefore, the water leak inspection system 1 of the present embodiment performs a water leak inspection in consideration of such an error.

<< Configuration of water leak inspection system 1 >>
The configuration of the water leak inspection system 1 of the present embodiment is the same as that of the first embodiment except for the image processing device 200.

FIG. 23 is a diagram showing an example of the configuration of the image processing device 200 according to the second embodiment. As shown in the figure, the configuration of the image processing apparatus 200 of the present embodiment is the same as that of the first embodiment, but the water droplet determination unit 220 of the image processing apparatus 200 further includes a visible light error data storage unit 227. Further, the residual moisture determination unit 230 of the image processing device 200 further includes a far-infrared error data storage unit 237.

The visible light error data storage unit 227 stores the region of the visible light image corresponding to the inspection error that occurs when the visible light image of the object 5 to be inspected 5 is acquired as the visible light image error region.
Utilizing this, the visible light mask region generation unit 221 sets a characteristic region of the specified visible light image determination region and the corresponding visible light image error region as the visible light mask region. ..

The far-infrared error data storage unit 237 stores the region of the far-infrared image corresponding to the inspection error that occurs when the far-infrared image of the object 5 to be inspected 5 is acquired as the far-infrared image error region.

Utilizing this, the far-infrared mask region generation unit 231 sets a characteristic region of the specified far-infrared image determination region and the corresponding far-infrared error region as the far-infrared mask region.

Information in the visible light image error region and the far infrared image error region is stored in the error information 260.

FIG. 24 is a diagram showing an example of error information 260. The error information 260 is the type of error applied to the target image 261 in which the information of the type of image (visible light image or far infrared image) to which the error is applied is stored and the image indicated by the target image 261. One or more having each item including an error type 263 in which information is stored and an error area 265 in which the specific content of the error indicated by the error type 263 (eg, a predetermined range of planes or spaces) is stored. It is a database of records.

The error type 263 includes, for example, an error in the housing of the inspected object 5, an error caused by the operation of the inspected object 5 performed during the inspection (vibration of the inspected object 5, etc.), and an inspected object in the transport device 160. It includes an error in the position of the object to be inspected 5 when installing the object 5, an error in the stop position of the object to be inspected 5 for photographing by the visible light camera 20 or the far-infrared camera 50, and the like.

<< Water leak inspection method >>
The process performed by the water leak inspection system 1 of the present embodiment is the same as that of the first embodiment except for the visible light mask data generation process and the far infrared mask data generation process. Therefore, the visible light mask data generation process and the far-infrared mask data generation process in the present embodiment will be described below.

(Visible light mask data generation process)
FIG. 25 is a flow chart illustrating an example of the visible light mask data generation process according to the second embodiment. First, the processing of s51 and s53 (generation of gradation difference information and identification of a characteristic region having a large gradation difference) is the same as that of the first embodiment.

Subsequently, the image processing apparatus 200 adds a visible light image error region to the characteristic region having a large gradation difference identified in s53 (s54).

Specifically, for example, the image processing apparatus 200 acquires all the records in which the visible light image is registered in the target image 261 from the error information 260, and s53 the area indicated by the error area 265 of the acquired records. Add to the area specified in.

After that, the image processing apparatus 200 generates and stores visible light mask data (s55-s61) based on the added region, as in the first embodiment.

(Far infrared mask data generation processing)
FIG. 26 is a flow chart illustrating an example of the far-infrared mask data generation process according to the second embodiment. First, the treatments of s91, s93, s95, and s96 (calculation of temperature distribution, specification of minimum temperature, calculation of average temperature of the surface of the object 5 to be inspected and average temperature of the pipe surface filled with water, threshold determination, and , Identification of a characteristic region with a low temperature) is the same as in the first embodiment.

Subsequently, the image processing apparatus 200 adds a far-infrared image error region to the low temperature region specified in s96 (s97).

Specifically, for example, the image processing apparatus 200 acquires all the records in which the far-infrared image is registered in the target image 261 from the error information 260, and s96 a region indicated by the error region 265 of the acquired records. Add to the area specified in.

After that, the image processing apparatus 200 generates and stores far-infrared mask data (s98-s101) based on the added region, as in the first embodiment. This completes the far-infrared mask data generation process.

The above description of the first and second embodiments is for facilitating the understanding of the present invention, and does not limit the present invention. The present invention can be modified and improved without departing from the spirit thereof, and the present invention includes its equivalents.

For example, in each embodiment, it is assumed that the object 5 to be inspected contains water, but the water leakage inspection system 1 detects water leakage by a far infrared image using the magnitude of the heat of vaporization of water. Therefore, for an object (inspected object 5) that operates using a liquid having a large heat of vaporization (for example, a liquid having a higher specific heat than the surface material of the inspected object 5, including a mixture with water). Can also be applied.

Further, in each embodiment, the object to be inspected 5 is a washing machine, but in addition to the washing machine, home appliances such as a dehumidifier, a humidifier, a water heater, and a floor heater, as well as automobile parts (radiator, etc.) ) Can be widely applied to devices that are used with water.

Further, the vaporization rate of water droplets (moisture) on the surface of the object to be inspected 5, which is an important element in the water leakage inspection system 1 of each embodiment, is variously affected by water temperature, air temperature, wind speed, humidity and the like. Therefore, a dehumidifying device may be newly provided according to the humidity of the first inspection area 120 and the second inspection area 140, or the temperature of water, the temperature of the first inspection area 120 or the second inspection area 140, or the subject may be covered by other devices. It is necessary to adjust the size (wind speed) of the air blown to the inspection object 5.

The above description of the present specification clarifies at least the following. That is, in the water leakage inspection system of the present embodiment, if water leakage does not occur in the region specified as the characteristic region of the visible light determination region, the characteristic region is newly added. It may be set as a visible light mask area.

Further, in the water leakage inspection system of the present embodiment, if water leakage does not occur in the area specified as the characteristic area of the far infrared ray determination area, the area is used as a new far infrared ray mask. It may be set as an area.

In this way, when it is erroneously detected that there is water leakage even though there is no water leakage, the visible light image or far infrared image in that case is converted into mask data and stored, so that the stored mask data thereafter. False positives can be reduced based on.

Further, in the water leakage inspection system of the present embodiment, the region of the visible light image corresponding to the inspection error that occurs when the visible light image of the object to be inspected is acquired is stored as the visible light image error region, and the region is stored as the visible light image error region. The characteristic region of the specified visible light image determination region and the corresponding visible light image error region may be set as the visible light mask region.

Further, in the water leakage inspection system of the present embodiment, the region of the far-infrared image corresponding to the inspection error that occurs when the far-infrared image of the object to be inspected is acquired is stored as the far-infrared image error region, and the region is stored as the far-infrared image error region. The characteristic region of the specified far-infrared image determination region and the corresponding far-infrared error region may be set as the far-infrared mask region.

In this way, an image area corresponding to an inspection error that occurs when the visible light image and far infrared rays of the object 5 to be inspected are acquired is stored as an error area, and this stored area is stored as a visible light image or a far infrared image. By adding it to the characteristic area related to the above, it is possible to perform image determination in consideration of errors that occur in the inspection, and it is possible to improve the accuracy of the water leak inspection.

Further, in the water leakage inspection system of the present embodiment, the inspection error is a shape error between the inspected objects and a displacement of the inspected object due to the operation of the inspected object performed during the inspection. It may include either the error of the above or the error of the displacement of the inspected object due to the transportation of the inspected object performed during the inspection.

The accuracy of the water leak inspection can be improved by considering such an error with a high frequency of occurrence that affects the image determination.

Further, in the water leakage inspection system of the present embodiment, a visible light mask area input unit that receives an input for setting the visible light mask area for the object to be inspected for which no water leakage has occurred on the surface thereof is provided. You may prepare.

Further, in the water leakage inspection system of the present embodiment, an input for setting the far-infrared mask region may be received from the user for the object to be inspected for which no water leakage has occurred on the surface thereof.

As described above, by having the function of accepting the input for setting the visible light mask area and the far infrared mask area from the user, the degree of freedom and flexibility of the inspection can be increased.

Further, in the water leakage inspection system of the present embodiment, the information of the visible light mask region set above may be output.

Further, in the water leakage inspection system of the present embodiment, the information of the far-infrared mask region set above may be output.

By outputting the information of the visible light mask area and the far-infrared mask area in this way, the user confirms the area of the inspected object 5 that causes erroneous detection, and enhances the speed and accuracy of the inspection. Can be done.

Further, in the water leak inspection system of the present embodiment, visible light images of water droplets falling from a predetermined position are acquired from the visible light camera at a plurality of timings, and the gradation difference changes in a predetermined direction with the progress of time. When it is determined that the acquired visible light image region is present, the water leakage inspection of the object to be inspected by the visible light image may be performed.

Further, in the water leakage inspection system of the present embodiment, a far-infrared image of the surface of a predetermined member having a surface that does not reflect far-infrared rays and water droplets are arranged on the surface is acquired from the far-infrared camera. When the average temperature of the surface is calculated, a particularly low temperature among the temperatures of the surface is specified, and a characteristic region where the difference between the specified low temperature and the average temperature is equal to or more than a predetermined value is specified, the above-mentioned A water leak inspection of the object to be inspected by far infrared rays may be performed.

In this way, it is checked in advance whether or not the water droplet 7 can be detected by using the visible light image of the water droplet falling from the predetermined position (nozzle 402), and the predetermined member (the water droplet is arranged on the surface thereof). By preliminarily checking whether water droplets (fine water droplets 8) can be detected using the far-infrared image of the surface of the test plate 404), water leakage due to the visible light image and the far-infrared image for the object 5 to be inspected and Residual water can be detected without error.

Further, in the water leakage inspection system of the present embodiment, the predetermined member may be an aluminum plate whose surface is anodized.

By arranging the water droplets on the surface of the aluminum plate whose surface has been anodized in this way, it is possible to perform a preliminary inspection with each water droplet (water droplet 7, fine water droplet 8) without error.

Further, in the water leakage inspection system of the present embodiment, the surfaces of the structure and the object to be inspected include the surface of the housing of the structure and the object to be inspected and the surface of the internal piping thereof, respectively. good.

In this way, by performing each of the above image processing on the surface of the housing and the surface of the internal piping, water leakage from the outside and the inside from the object to be inspected can be detected without error.

1 Water leak inspection system, 5 Inspected object, 20 Visible light camera, 50 Far infrared camera, 221 Visible light mask area generation unit, 223 Visible light image judgment unit, 231 Far infrared mask area generation unit, 233 Far infrared image judgment unit

Claims (8)

A water leak inspection system that determines water leakage from the surface of an object to be inspected that can contain water, and the water leak inspection system is
Visible light camera and
It is equipped with an information processing device that is communicably connected to the visible light camera.
The information processing device is
In visible light learning mode
From the visible light camera, first visible light images of the surface of the structure having substantially the same structure as the object to be inspected and having no water leakage on the surface are acquired at a plurality of timings.
Based on each acquired first visible light image, a characteristic region of the first visible light image in which the gradation difference extends with the progress of time is identified.
The specified characteristic area and its surrounding area are set as a visible light mask area, and the area is set as a visible light mask area.
In visible light inspection mode
Second visible light images of the surface of the object to be inspected containing water are acquired at multiple timings.
Each of the acquired second visible light images excluding the portion corresponding to the visible light mask area is specified as a visible light determination area.
Based on each identified visible light determination region, when a characteristic region of the visible light determination region in which the gradation difference extends with the progress of time is specified, the inspected object containing the water is contained. Judge that a water leak has occurred in
Water leak inspection system. The water leak inspection system is
There is a region of the acquired visible light image in which visible light images of water droplets falling from a predetermined position are acquired from the visible light camera at a plurality of timings and the gradation difference is extended with the progress of time. The water leak inspection system according to claim 1, wherein the water leak inspection of the inspected object is performed by the visible light image when it is determined to be so. The water leak inspection system is
The first aspect of the present invention is to set the characteristic region as a new visible light mask region when water leakage does not occur in the region specified as the characteristic region of the visible light determination region. Water leak inspection system. The water leak inspection system is
The area of the second visible light image corresponding to the inspection error that occurs when the second visible light image of the object to be inspected is acquired is stored as the visible light image error area.
The water leak inspection system according to claim 1, wherein a characteristic region of the specified first visible light image and a corresponding visible light image error region are set as the visible light mask region. The water leak inspection system is
The water leakage inspection system according to claim 1, wherein an input for setting the visible light mask region is received from a user for an object to be inspected for which no water leakage has occurred on the surface thereof. The water leak inspection system is
The water leakage inspection system according to claim 1, which outputs information on the set visible light mask area. The error in the inspection is an error in the shape between the objects to be inspected, an error in displacement of the object to be inspected due to the operation of the object to be inspected during the inspection, or the error to be performed during the inspection. The water leak inspection system according to claim 4 , which comprises any of the displacement errors of the inspected object due to the transportation of the inspected object. The water leak inspection system according to claim 1 , wherein the surfaces of the structure and the object to be inspected include the surface of the housing of the structure and the object to be inspected and the surface of the internal piping thereof, respectively.

Images