JP2002366953A - Method, device and program for extracting image, and method, device and program for detecting abnormality of pole mounted equipment of distribution pole utilizing image extracting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To unnecessitate strict positioning between an infrared camera and a visible light camera. SOLUTION: This image extracting device is provided with a distance image preparing means 6 for preparing a distance image 5 in which a color is changed for each distance on the basis of distance information calculated from a plurality of visible light images 4 obtained by photographing an object 2, a correspondence point extracting means 7 for calculating correspondence points of a thermal image 3 of the object 2 and the distance image 5, a matrix preparing means 8 for calculating a conversion matrix that coincides the correspondence point of the distance image 5 with that of the thermal image 3, an image coinciding means 9 for specifying the object 2 in the distance image 5 and also converting either an area including the object 2 of the distance image 5 or the thermal image 3 with the conversion matrix before or after this specification to coincide both images with each other, and an image extracting means 10 for extracting the object 2 from the thermal image 3 by utilizing the position of the object 2 in the coincided distance image 5'.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to an image extraction method and apparatus, an image extraction program, and a method and apparatus for detecting an abnormality of equipment on a pole of a distribution pole using the image extraction method, and an abnormality detection program. More specifically, the present invention relates to an image extraction method and apparatus using a thermal image and a distance image of an object, an image extraction program, and a method and an apparatus for detecting an abnormality of equipment on a power pole using the image extraction method, and an abnormality. Regarding the detection program.

[0002]

2. Description of the Related Art A thermal image taken by an infrared camera is sometimes used to measure the temperature of an object. However, since there are many similar thermal patterns in the thermal image alone, it is very difficult to extract the measurement target from the thermal image, which hinders high precision and automation of temperature measurement. Therefore,
Techniques have been developed for extracting an object in a thermal image by using a visible image obtained by photographing the object with a visible light camera.

At this time, if the visual field of the thermal image and that of the visible image are the same, the position coordinates of the object in the visible image can be directly projected on the thermal image, so that the positioning of the object in the thermal image can be easily performed. Can be.

For this reason, an image captured using one photographing lens is separated by a half mirror and photographed by an infrared camera and a visible light camera so that each camera can photograph an image having the same field of view. An object extraction device has been developed (see Japanese Patent Application Laid-Open No. 8-189809).

[0005]

However, in the above-described object extracting apparatus, since an image in the same field of view must be taken with an infrared camera and a visible light camera, the half mirror and each camera can be accurately positioned with respect to the taking lens. Position, and a complicated adjustment operation such as optical axis alignment is required before the measurement is started.

Further, it is necessary to increase the rigidity of the assembly so that the positions of the cameras do not shift from each other due to vibration or the like during the measurement, so that the object extracting apparatus is increased in size and weight.

Accordingly, the present invention provides an image extracting method and apparatus which can eliminate the need for precise alignment between an infrared camera and a visible light camera, an image extracting program, and a method for detecting an abnormality of equipment on a power pole using the image extracting method. And an apparatus and an abnormality detection program.

[0008]

In order to achieve the above object, an image extracting method according to claim 1 is a method for extracting a distance from a plurality of visible images obtained by photographing an object by changing a color for each distance. A distance image creating step of creating an image, a corresponding point extracting step of finding corresponding points of the thermal image and the distance image of the object, a matrix creating step of finding a transformation matrix that matches the corresponding points of the distance image and the thermal image, An image matching step of specifying an object in the image and converting one of the region including the object and the thermal image of the distance image before or after the specific image by a conversion matrix to match the two images, and And an image extracting step of extracting the object from the thermal image using the position of the object.

According to a third aspect of the present invention, there is provided an image extracting apparatus, comprising: a distance image creating means for creating a distance image in which a color is changed for each distance based on distance information obtained from a plurality of visible images of an object; Corresponding point extracting means for finding corresponding points of the thermal image and the distance image of the object; matrix creating means for finding a transformation matrix for matching the corresponding points of the distance image and the thermal image; and identifying and identifying the object in the distance image Image matching means for converting one of the region including the object in the distance image and the thermal image before or after by using the conversion matrix to match the two images, and a thermal image using the position of the object in the distance image after matching And an image extracting means for extracting an object from the image.

Therefore, one of the region including the object and the thermal image in the range image is converted by the conversion matrix to match the two images, and then the object is cut out from the thermal image. There is no need to match the field of view of each camera when capturing images. This eliminates the need for precise positioning of the infrared camera and the visible light camera, thereby facilitating camera positioning. In addition, since high rigidity of the image extraction device including the camera is not required, the size and weight of the device can be suppressed.

Further, according to a fourth aspect of the present invention, there is provided an image extraction program for controlling a computer to extract a thermal image of an object using the thermal image of the object and a plurality of visible images. A distance image creating means for creating a distance image in which a color is changed for each distance based on the distance information obtained from the image, a corresponding point extracting means for finding corresponding points of the thermal image of the object and the distance image, and a distance image and the thermal image Matrix generation means for obtaining a conversion matrix that matches the corresponding points of the distance image, and specifying an object in the distance image and converting one of a region including the object of the distance image and the thermal image before or after the specific image by a conversion matrix. Image matching means for matching the two images
After the matching, the position of the object in the distance image is used to function as an image extracting means for extracting the object from the thermal image.

Therefore, by executing the program on the computer, the object can be extracted from the thermal image without strictly aligning the infrared camera and the visible light camera.

The invention according to claim 2 is the invention according to claim 1.
In the described image extraction method, in the image matching step, the distance image is converted by a conversion matrix to match the thermal image, and then the target object is specified in the converted distance image. Therefore, since the object is cut out from the thermal image after the distance image and the thermal image are matched, it is not necessary to adjust the field of view of each camera when capturing the thermal image and the visible image, which facilitates the work of aligning the cameras. Can be

On the other hand, according to a fifth aspect of the present invention, there is provided a method for detecting abnormalities of pole equipment on a power pole, wherein a power pole equipped with an arm and pole equipment provided on the arm is photographed by one infrared camera and a thermal image is taken. 2. Obtaining a stereo image with at least two visible light cameras, obtaining one visible image from each visible light camera, and obtaining the obtained thermal image and the plurality of visible images.
To extract an object from the thermal image by processing according to the image extraction method according to any one of the above to create a thermal image after extraction;
An image of the arm is extracted from the post-extraction thermal image, and based on the position of the arm, a temperature gradient on the pole equipment and its surroundings is calculated to determine whether the pole equipment is normal or abnormal.

According to a sixth aspect of the present invention, there is provided an apparatus for detecting abnormalities of pole equipment on a power pole, wherein one infrared camera and at least two infrared cameras for photographing a power pole equipped with an arm and pole equipment provided on the arm. 4. An image according to claim 3, wherein an object is extracted from the thermal image based on the visible light camera, the thermal image obtained by the infrared camera, and the plurality of visible images obtained by the visible light camera, and the extracted thermal image is created. An extracting device, an arm extracting means for extracting an image of the arm from the extracted thermal image, a temperature gradient calculating means for calculating a temperature gradient on the pole and its surroundings based on the position of the arm, and a pole on the pole from the temperature gradient Abnormality determining means for determining whether the is normal or abnormal.

Further, according to a seventh aspect of the present invention, there is provided a program for detecting an abnormality of a pole on a pole of a distribution pole, wherein the distribution pole including an arm and a pole on the arm is photographed with a single infrared camera to obtain a thermal image. And at least two visible light cameras to perform stereo photography to obtain one visible image from each visible light camera, and use the obtained thermal image and a plurality of visible images to determine whether the pole equipment is normal. A distance image creating means for creating a distance image in which a color is changed for each distance based on distance information obtained from a plurality of visible images, and a method for determining corresponding points of the thermal image and the distance image in order to determine whether the image is abnormal Point extraction means, matrix creation means for obtaining a transformation matrix for matching the corresponding points of the distance image and the thermal image, an area specifying the object in the distance image and including the object of the distance image before or after the specific Image matching means for converting one of the thermal image and the thermal image by a conversion matrix to match the two images, and extracting the thermal image from the thermal image using the position of the object in the distance image after the matching, and extracting the extracted thermal image. An image extraction unit to be created, an arm extraction unit to extract an image of the arm from the post-extraction thermal image, a temperature gradient calculation unit to calculate a temperature gradient of the on-board equipment and its surroundings based on the position of the arm, and It is designed to function as abnormality determination means for determining whether the pole equipment is normal or abnormal.

Therefore, the work of positioning the camera for photographing the pole equipment can be facilitated and the size and weight of the apparatus can be suppressed, so that the abnormality determination of the pole equipment can be easily performed. become. In addition, since no artificial work is required from photographing the equipment on the pole to detecting an abnormality, the entire process can be automated.

[0018]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The structure of the present invention will be described below in detail based on one embodiment shown in the drawings. FIG. 1 shows an example of an embodiment of an image extraction device 1 of the present invention. The image extracting apparatus 1 extracts an object 2 from a thermal image 3 using a thermal image 3 and a plurality of visible images 4 of the object 2. The image extracting apparatus 1 includes a distance image creating unit 6 that creates a distance image 5, a corresponding point extracting unit 7 that finds corresponding points of the thermal image 3 and the distance image 5, a matrix creating unit 8 that finds a transformation matrix H, An image matching unit 9 for matching the region including the object 2 of the distance image 5 with the thermal image 3 and an image extracting unit 10 for extracting the object 2 from the thermal image 3 after matching are provided. For this reason, the region including the object 2 in the distance image 5 and the thermal image 3 are matched, and then the thermal image 3
Since the target object 2 has been cut out, a plurality of visible images 4
In addition, it is not necessary to match the fields of view of the cameras 11 and 12 when capturing the thermal image 3. Therefore, since it is not necessary to precisely align the infrared camera 11 and the visible light camera 12, the work of aligning the cameras 11 and 12 can be facilitated. Further, the image extracting device 1 including the cameras 11 and 12
Since high rigidity is not required, the size and weight of the apparatus 1 can be suppressed.

The distance image creating means 6 creates a distance image 5 (see FIG. 3) having a different color for each distance based on distance information obtained from a plurality of visible images 4 (see FIG. 2) of the object 2. Shall do. For example, two visible light cameras 12a and 12b are connected to the distance image creating means 6. Then, the same object 2 is photographed from slightly different photographing directions using the cameras 12a and 12b, and each object on the visible image 4 is photographed by the distance image creating means 6 using the plurality of visible images 4 obtained. Is calculated. The distance image 5 is obtained by coloring an object in the image in correspondence with the distance from the camera 12, and is created such that, for example, as shown in FIG. I do.

The corresponding point extracting means 7 extracts feature points from the thermal image 3 (see FIG. 4) and the distance image 5,
The five feature points are associated with each other. These features include:
A quadratic curve (conic) included in an artifact is used, particularly the center of a circle or ellipse. Since the thermal image is obtained by visualizing the radiation information of the temperature, highly accurate measurement can be performed by using a circle or an ellipse.

The matrix creating means 8 finds a transformation matrix H for matching the corresponding points of the distance image 5 and the thermal image 3. This transformation matrix H is called a so-called homography, and is defined as follows.

As shown in FIG. 7, the target point on a certain plane π
X_πIs projected on each of the distance image 5 and the thermal image 3.
The corresponding point in the distance image 5 is X₁, Thermal image 3
X₂And Then, in projective geometry, X₁=
H_1πX_π, X₂= H_2πX _πCan be expressed as these
X from relationship_πIs deleted, X₂= H_2πH_1π ^-1X
₁= HX₁Is derived. And the strange here
Permutation matrix H = H_2π/ H _1πIs called homography.

This X ₂ = HX ₁ is specifically shown in Equation 1.

(Equation 1) Here, X ₁ (x ₁ , y ₁ ) and X ₂ (x ₂ , y ₂ ) are points in both images, and f ₁ and f ₂ are cameras 1 that have captured each image.
These are the focal lengths of the lenses 1 and 12.

Since the transformation matrix H is a 3.times.3 matrix, it is sufficient to determine combinations of nine or more corresponding points in order to determine each component.

The image matching means 9 converts and projects the distance image 5 by using the conversion matrix H so as to match the thermal image 3 to obtain a distance image 5 'shown in FIG. Then, the target object 2 is specified based on the distance information in the projected distance image 5 ′.

The image extracting means 10 transfers the position of the object 2 in the distance image 5 'to the thermal image 3 as it is,
Then, the target object 2 is extracted as shown in FIG. In the present embodiment, an image obtained by extracting the object 2 is referred to as an extracted thermal image 13.

The above-described image extracting apparatus 1 is used, for example, by being mounted on an abnormality detecting apparatus 16 of pole equipment 15 of a distribution pole 14. As shown in FIG. 1, the abnormality detection device 16 includes one infrared camera 11 and at least two visible light cameras for photographing a distribution pole 14 including an arm 17 and a pole device 15 provided on the arm 17. 12
Based on a and 12b, the thermal image 3 obtained by the infrared camera 11, and the plurality of visible images 4 obtained by the visible light camera 12, the arm 17 and the pole equipment 15 as the object 2 are extracted from the thermal image 3. The image extracting apparatus 1 according to claim 4, wherein the post-extraction thermal image 13 is created, an arm extracting means 18 for extracting an image of the arm 17 from the post-extraction thermal image 13, and a pole-mounted device 15 based on the position of the arm 17. And a temperature gradient calculating means 19 for calculating a temperature gradient in the vicinity thereof, and an abnormality determining means 20 for determining whether the pole equipment 15 is normal or abnormal based on the temperature gradient. Further, the arm extracting means 18 is provided with a teacher image creating means 21 for creating a teacher image 26 of the arm 17 and its feature vector T in advance.

For this reason, the work of aligning the cameras 11 and 12 for photographing the pole equipment 15 can be facilitated, and the size and weight of the device 16 can be suppressed. It can be easily performed. In addition, since no artificial work is required from photographing of the pole device 15 to detection of an abnormality, the entire process can be automated.

The image extracting apparatus 1, the arm extracting means 18, the teacher image creating means 21, the temperature gradient calculating means 19, and the abnormality judging means 20 are constituted by a computer system. Although not shown, the computer system includes, for example, a central processing unit that executes various types of arithmetic processing, a main storage device that stores programs and data required for executing the arithmetic processing, and an input / output port. Various interfaces such as distance image creating means 6, corresponding point extracting means 7, matrix creating means 8, image matching means 9, image extracting means 10, arm extracting means 18, teacher image creating means 21, temperature gradient calculating means 19, For example, a hard disk drive for storing a program for realizing the abnormality determining means 20 and various data and constants necessary for executing the processing, and an input for inputting photographing data from the cameras 11 and 12 and necessary commands It is composed of a device and an output device for outputting a result processed by the central processing unit.

The procedure for executing the abnormality detection of the pole equipment 15 by the abnormality detection device 16 of the pole equipment 15 of the distribution pole 14 will be described with reference to the flowcharts shown in FIGS. 8, 9 and 15.

At the same time as the two visible light cameras 12a and 12b perform stereo photography of the power distribution pole 14, the one infrared camera 11 also photographs the power distribution pole 14 (step 1). 2 photographed by the visible light cameras 12a and 12b
The visible images 4 and 4 are input to the distance image creating means 6, and a distance image 5 is created based on these images 4 and 4 (step 2).

Then, the thermal image 3 is output by the corresponding point extracting means 7.
The feature points are extracted from the distance image 5 and the two images 3 and 5.
The correspondence between the feature points is performed (step 3). The association of the feature points is specifically performed as shown in the flowchart of FIG. Here, the procedure for extracting feature points from both images 3 and 5 will be described as an example by replacing the procedure for extracting feature points from original image 22 shown in FIG. The procedure will be described.

First, the original image 22 is convoluted with a Gaussian filter, and a group of points whose obtained value is equal to or greater than a certain threshold is selected, and a line in the image is thinned (step 2).
1) Extract edges (step 22) to obtain an edge image 23 shown in FIG. Then, a conic candidate having a certain length is extracted from the edge image 23 (step 2).
3), a conic candidate image 24 shown in FIG. 12 is obtained.

Further, for each conic candidate, the coefficients A, B, C,
D, E, and F are calculated (step 24).

Ax ² + 2Bxy + Cy ² +2 (Dx + Ey)
+ F = 0 Here, x and y are the coordinates on the image of each candidate. Those obtained from the obtained coefficient set satisfying the condition of a circle or an ellipse of AC-B ² > 0 (step 25). As a result, a conic image 25 shown in FIG. 13 is obtained by extracting a large curvature from the conic candidate images 24 as a circle or an ellipse.

Further, the center point of a circle or ellipse appearing in each of the thermal image 3 and the distance image 5 is used as a feature point, and in the case of an ellipse, the major and minor axes are extracted (step 26). Then, matching is performed between the images 3 and 5 with respect to the center point of the circle or the ellipse or the end point of the long and short axes of the thermal image 3 and the distance image 5 (step 27). As a result of the matching, those that have a correspondence without contradiction are defined as corresponding points.

In FIG. 13, a vertically long ellipse on the left side and a horizontally long ellipse on the lower right side are not included in the original image 22, and therefore need to be excluded as ellipses for determining feature points. For such circles and ellipses to be excluded, the center point is once raised as a feature point, and the matching process is performed so that feature points are corresponded between the images 3 and 5 so as to be removed when inconsistency arises. ing.

Also, the corners of a figure or the intersections of straight lines can be used as feature points. However, if the feature points are the intersections of the corners of the figure or the straight line, it is difficult to take correspondence without inconsistency when taking correspondence between the feature points between the images 3 and 5 because of the large number. The inconsistency is likely to occur because there are too few clues to take the countermeasure. It is considered that if the clues are sufficient, the inconsistency hardly occurs and the accuracy of taking the countermeasure is improved. Thus, in the present embodiment, attention is paid to the fact that circles and ellipses do not appear so much in an image, and correspondence is made with the center point of the circle or ellipse as a feature point. According to this, since sufficient clues can be obtained for taking a correspondence with respect to one feature point, it is possible to improve the estimation accuracy of the correspondence.

On the basis of the correspondence obtained by the corresponding point extracting means 7, a transformation matrix H for matching the corresponding points of the distance image 5 and the thermal image 3 is obtained by the matrix creating means 8. further,
The image matching means 9 converts the distance image 5 by the conversion matrix H so as to match the thermal image 3. Then, the target object 2 is specified in the converted distance image 5 based on the distance information. The object 2 is specified by, for example, determining the object closest to the cameras 11 and 12 as the object 2. After specifying the target object 2, the target object 2 is cut out from the thermal image 3 by the image extracting means 10 to create a post-extraction thermal image 13.

On the other hand, on-column equipment 15 for which abnormality is to be detected,
For example, a teacher image 26 of the arm 17 to which three pin insulators are attached and its feature vector T are created in advance by the teacher image creating means 21. The feature vector T is obtained as a feature pattern using a so-called higher-order local autocorrelation feature for the teacher image.

Then, the arm extracting means 18
As shown in FIG. 4, one pixel of the extracted thermal image 13 extracted by the image extracting device 1 is selected, and the selected pixel 27 is set to the arm 17.
Are matched (step 31).
Further, the teacher image 26 of the arm 17 is rotated up and down by one degree, for example, ± 15 degrees around the selected pixel 27 (step 32). That is, the teacher image 26 can be located at 31 positions with respect to one selected pixel 27. When the teacher image 26 is at each position, matching is obtained by performing matching between the teacher image 26 and the extracted thermal image 13 at a position overlapping the teacher image 26 (step 33). This matching process is performed by comparing the feature vectors of both images. As a comparison method, various known methods conventionally used,
Alternatively, a new method can be adopted. Thereby, 31 coincidences are calculated for one selected pixel 27.

Further, the selected pixel 27 is changed, and the matching process between the teacher image 26 and the whole image is performed again. Here, the selection order of the selection pixels 27 is, for example, as shown in FIG. 14, lowering from the upper left pixel, and when reaching the lowermost pixel, the selection pixel 27 is raised to the uppermost pixel in the adjacent column and lowered again. However, it goes without saying that the selection order of the selected pixels 27 is not limited to this. In addition, the selected pixel 27 is the extracted thermal image 13
When the teacher image 26 is within a few pixels from above, if the teacher image 26 is rotated upward, it will come out of the extracted thermal image 13 and an accurate matching process cannot be performed, so the rotation process may be omitted. Similarly, when the selected pixel 27 is within several pixels from the bottom of the extracted thermal image 13, the process of rotating the teacher image 26 downward may be omitted. Furthermore, when the selected pixel 27 is within a range shorter than the teacher image 26 from the right end of the extracted thermal image 13, the matching process may be omitted because the right end of the teacher image 26 jumps out of the entire image.
By omitting such unnecessary processing, the overall processing speed can be increased.

Then, it is determined whether or not matching processing has been performed for all the selected pixels 27 as candidates (step 3).
4) If it is not yet (step 34; NO), the next selected pixel 27 is selected and the process is continued (steps 31 to 3).
3). When the matching process is completed for all the selected pixels 27 of the candidates (Step 34; YES), the one having the highest value is selected from all the matching degrees. Then, it is determined that the image of the arm 17 is located at the position of the teacher image 26 having the highest matching degree. Based on the image of the arm 17, the inclination angle θ of the arm 17 and the positions of both ends are detected (step 35).

Also, one power pole 14 is usually provided with one arm 17, and an image other than the power pole 14 of the object 2 is deleted from the thermal image 3 taken by the image extracting device 1. Therefore, there is usually one arm 17 in the image input to the arm extracting means 18. Therefore, the accuracy of extracting the arm 17 from the image can be kept high.

Next, the temperature gradient calculating means 19 scans each pixel along the arm 17 in the direction Y orthogonal to the arm 17 as shown in FIG. (Hereinafter, referred to as a temperature gradient). Here, the thermal image 3 is obtained by visualizing the temperature data obtained by the infrared camera 11 with, for example, 256 levels of luminance. Therefore, a higher temperature resolution can be obtained by using the value of the temperature data directly obtained by the infrared camera 11 than by using the luminance value obtained from the thermal image 3. Therefore, in the present embodiment, in order to detect the temperature of the pole equipment 15 and its surroundings, the temperature data corresponding to each pixel is used without using the luminance value corresponding to each pixel of the thermal image 3. .

Specifically, as shown in FIG. 16, seven pixels are set from one pixel on the extracted arm 17 to 30 pixels every five pixels in the direction Y orthogonal to the arm 17. Then, a pixel set of 5 × 5 pixels centering on each of the seven pixels is set, and the averages I _{1 to} I ₇ of the temperatures corresponding to each pixel set are obtained. Further, an average temperature difference between adjacent ones of the set of pixels 7 places, the cumulative absolute value of the difference (temperature gradient) as shown in Equation 3, the cumulative temperature gradient T _x.

(Equation 3) This cumulative temperature gradient _Tx is obtained for all pixels along the arm 17. Then, as shown in FIGS. 17 to 19, determining the relationship between the cumulative temperature gradient T _x in the arm direction of the pixel values and the pixel.

Next, the arm direction shown in FIGS.
Pixel value and cumulative temperature gradient T at each pixel_xRelationship Gra
And the on-pillar equipment 15
The maximum cumulative temperature gradient T_xFind the difference between Meanwhile, the figure
As shown in FIG. 20, the actual temperature of
Difference and maximum cumulative temperature gradient T of healthy equipment and defective equipment _x
Find the relationship with the difference. This relationship is
Depending on the resolution of the
Since they are different, they are determined before starting the measurement.

In this embodiment, when the value of the difference of the maximum cumulative temperature gradient is 10 ° C. or more, it is determined that the pole equipment 15 having the larger maximum cumulative temperature gradient is abnormally heated and malfunctions. ing. That is, as shown in FIG. 20, the pole equipment 15 whose temperature is higher than the healthy equipment by 2 ° C. or more is determined to be defective equipment.

FIGS. 17 and 18 _show the relationship between the pixel value in the arm direction and the cumulative temperature gradient Tx at each pixel.
Is obtained, the difference between the maximum cumulative temperature gradients of the three pin insulators is within 10 ° C., so that it can be determined that all of them are sound equipment.

[0049] Further, as the relationship of the cumulative temperature gradient T _x in the arm direction of the pixel values and the pixel, for example, if the result is obtained as shown in FIG. 19, the maximum cumulative temperature gradient in the center of the pin insulator is across pins insulators 10 compared to the maximum cumulative temperature gradient of
Since the temperature is higher than ℃, it can be determined that the center pin insulator is defective.

The above embodiment is an example of a preferred embodiment of the present invention, but the present invention is not limited to this, and various modifications can be made without departing from the spirit of the present invention. For example, in the present embodiment, when the value of the difference of the maximum cumulative temperature gradient is equal to or more than 10 ° C., the pole device 15 having the larger maximum cumulative temperature gradient is determined as the defective device. Is not limited to 10 ° C. or more, and may be another value such as 5 ° C. The camera in the difference between the maximum cumulative temperature gradient T _x of the actual temperature difference and sound equipment and bad equipment sound gear and bad gear in this embodiment although the relationship shown in FIG. 20, to which the shooting not limited 1
Depending on the resolution of 1, 12 or the height of the pole equipment 15, another relationship may be obtained. That is, the relationship shown in FIG. 20 is only an example, the difference between the actual device 16 and the object 2 maximum accumulated temperature gradient T _x of the actual temperature difference and sound equipment and bad equipment previously healthy gear and bad gear in accordance with the Seeking a relationship with
The difference between the maximum cumulative temperature gradient T _x so as setting the threshold for determining whether the bad gear if what extent.

In the above embodiment, the onboard equipment 15
Although the image of the pin insulator is extracted as described above, the invention is not limited to this, and an image of a tension insulator, a switch, or the like may be extracted. Also in these cases, since the pole equipment 15 is attached to the arm 17, image extraction and abnormality determination of the pole equipment 15 can be performed with high accuracy and automation thereof can be promoted.

Further, in the above embodiment, the teacher image 2
6 and the extracted thermal image 13 are compared by comparing feature vectors of higher-order local autocorrelation features.
The present invention is not limited to this, and a known or new image matching method can be used. Also in this case, camera 1
In addition to making the installation of the first and the second equipment 12 easier, the measurement can be easily performed, and the image of the pole equipment 15 can be extracted without requiring any artificial work.

In the above-described embodiment, the image matching means 9 converts the distance image 5 so as to match the thermal image 3 by the conversion matrix H, and then specifies the object 2 in the converted distance image 5 '. However, the order is not limited to this. Even if the target 2 is specified in the distance image 5 before conversion, the target 2 is converted by the conversion matrix H so as to match the thermal image 3. good. Alternatively, in the above-described embodiment, the distance image 5 is converted by the conversion matrix H so as to match the thermal image 3. However, the present invention is not limited to this, and the thermal image 3 matches the distance image 5 by the conversion inverse matrix H ^-1 . The target object 2 may be identified by the unconverted distance image 5. In this case, the position of the object 2 in the distance image 5 is transferred to the converted thermal image in the image extracting means 10, and the object 2 is extracted from the thermal image.

In the above embodiment, the temperature gradient is calculated by the temperature gradient calculating means 19 using the temperature data. However, the present invention is not limited to this, and the luminance value of the thermal image 3 may be used as it is. . In this case, even when only the thermal image 3 can be used without obtaining the temperature data, the temperature gradient can be obtained.

Further, in each of the above-described embodiments, the image extracting device 1 and the abnormality detecting device 16 are configured by computer systems, but are not limited thereto. For example, it can be executed by operating a general-purpose computer using a program for causing a computer to execute the above-described image extraction method and abnormality detection method.

Here, the image extracting program includes a distance image creating means 6, a corresponding point extracting means 7, and a matrix creating means 8.
And an image matching unit 9 and an image extracting unit 10. Further, the abnormality detection program includes a distance image creating means 6, a corresponding point extracting means 7, a matrix creating means 8, an image matching means 9, an image extracting means 10, an arm extracting means 18, a temperature gradient calculating means 19 And function as the abnormality determination means 20.

In these cases, the program is executed by a general-purpose computer. However, the present invention is not limited to this, and the method may be executed by an independent device including a dedicated computer. Further, in the above-described embodiment, the computer executes the image extracting method. However, the present invention is not limited to this. The image extracting method may be executed by a sequence circuit. In any case, the installation of the cameras 11 and 12 can be facilitated and the measurement can be performed easily, and the automation of the image extraction of the pole equipment 15 can be promoted, which can contribute to the realization of an automatic power distribution equipment inspection device.

[0058]

EXAMPLE 1 Using the abnormality detection device 16 shown in FIG. 1, the cumulative temperature gradient of the object 2 shown in FIG. 21 was calculated. As shown in the figure, the distance between the cameras 11 and 12 and the object 2 is 3 m, and the distance between the object 2 and the background object 28 is 1.5 m.

The object 2 and the background object 28 were all three pillar-mounted devices 15. Further, the on-column equipment 15 in the center of the object 2 and the on-column equipment on the right side of the background object 28 are:
The temperature was set to be 3.6 ° C higher than other on-pole equipment.

The thermal image 3 taken at this time is shown in FIG. 4, the visible image 4 is shown in FIG. 2, the created distance image 5 is shown in FIG. 3, the converted distance image 5 'is shown in FIG. Images 13 are shown in FIG. In addition, the measurement result is indicated by the line “with distance information” in FIG. As shown in the figure, it is detected that only the on-column equipment 15 at the center of the target object 2 has a high temperature without being affected by the background object 28.

(Comparative Example) As in the first embodiment, the photographing of the object 2 and the calculation of the temperature gradient shown in FIG. 21 were performed using the abnormality detection device 16 shown in FIG. The difference from the first embodiment is that an image of the object 2 is extracted by the image extracting device 1 without using the distance information.

The measurement result is shown by the line “no distance information” in FIG. As shown in the drawing, the influence of the background object 28 has been detected, and it has been detected that the on-center equipment on the center and the right side has a high temperature.

Therefore, according to the abnormality detecting device 16 of the present invention, since the object 2 is extracted using the distance information in the image extracting device 1 as in the first embodiment, the influence of the background is minimized. can do.

(Embodiment 2) The abnormality detection device 16 shown in FIG.
Was used to calculate the cumulative temperature gradient of the object 2 shown in FIG. The distance between the cameras 11 and 12 and the object 2 is 3 m
And In addition, the object 2 was made up of three pillar-mounted devices 15.
Furthermore, a tape-shaped heater 29 was attached to the on-column equipment 15 at the center of the object 2, and the temperature was controlled by a sliderac 30.

FIGS. 17 to 19 show the results of the cumulative temperature gradient calculation.
Shown in FIG. 17 shows no heating, that is, when all the three pillar-mounted devices 15 are at room temperature, and FIG.
FIG. 19 shows the case of heating at 1.2 ° C., respectively. Then, showing the relationship between the difference between the maximum cumulative temperature gradient T _x of the actual temperature difference and sound equipment and bad equipment sound gear and bad gear calculated from these relationships in Figure 20.

As shown in FIG. 17, all of the on-post equipment 15
Are healthy, they do not always become equal due to variations in the accumulated temperature gradients. According to the present embodiment, this variation was about 3 ° C. 18 and FIG.
Referring to FIG. 9, the maximum cumulative temperature gradient T
_When the difference in _x is 10 ° C. or more, a clear temperature difference appears for the actual healthy equipment.

Therefore, the maximum cumulative temperature gradient T with the healthy equipment
It is considered appropriate to judge that _x is abnormal when the difference of _x is 10 ° C. or more. At this time, it is determined from FIG. 20 that the temperature difference between the healthy equipment and the defective equipment is 2 ° C. or more.

(Embodiment 3) The abnormality detecting device 16 shown in FIG.
Was used to calculate the cumulative temperature gradient of the object 2 shown in FIG. The distance between the cameras 11 and 12 and the object 2 is 3 m
And In addition, the object 2 was made up of three pillar-mounted devices 15.
Furthermore, a tape-shaped heater 29 was attached to the on-column equipment 15 at the center of the object 2, and the temperature was controlled by a sliderac 30. And the warm air blower 31 was installed behind the object 2.
This is to make the temperature behind the object 2 uniform.

[0069] From the results of the cumulative temperature gradient calculation obtained to determine the relationship between the difference between the background and the maximum accumulated temperature gradient T _x of the actual temperature difference and background and poor equipment failure gear. FIG. 25 shows the result.

[0070] As shown in the figure, when the difference between similarly maximum accumulated temperature gradient T _x to that of Example 2, and it is sufficient 10 ° C. or higher, the temperature difference above 0.8 ° C. The background and poor equipment It turned out that it could be distinguished. Therefore, even if the heat generation of the defective device is only about 0.8 ° C. lower than the background, the abnormality determination can be performed without being greatly affected.

[0071]

As is apparent from the above description, according to the image extracting method according to claim 1 or 2, the image extracting apparatus according to claim 3, and the image extracting program according to claim 4, a plurality of visible images can be obtained. In addition, it is not necessary to match the fields of view of the cameras when capturing a thermal image. This eliminates the need for precise positioning of the infrared camera and the visible light camera, thereby facilitating camera positioning. In addition, since high rigidity of the image extraction device including the camera is not required, the size and weight of the device can be suppressed.

On the other hand, a method for detecting an abnormality of a pole on a distribution pole according to claim 5, a device for detecting an abnormality of a pole on a distribution pole according to claim 6, and an abnormality of a pole on a distribution pole according to claim 7. According to the detection program, the work of positioning the camera for photographing the pole equipment can be facilitated, and the size and weight of the apparatus can be suppressed, so that the abnormality determination of the pole equipment can be easily performed. Become like Moreover,
Since no artificial work is required from photographing the equipment on the pole to detecting an abnormality, the entire process can be automated.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an image extraction device and an abnormality detection device for pole equipment on a distribution pole according to the present invention.

FIG. 2 is a diagram showing a visible image.

FIG. 3 is a diagram showing a distance image.

FIG. 4 is a diagram showing a thermal image.

FIG. 5 is a diagram showing a converted distance image.

FIG. 6 is a diagram showing a thermal image after extraction.

FIG. 7 is a conceptual diagram for obtaining a transformation matrix.

FIG. 8 is a flowchart showing a method for extracting an image and a method for detecting an abnormality of a pole equipment of a distribution pole according to the present invention.

FIG. 9 is a flowchart illustrating a processing procedure in an image matching step.

FIG. 10 is a diagram showing a visible image.

FIG. 11 is a diagram showing an edge image.

FIG. 12 is a diagram showing a conic candidate image.

FIG. 13 is a diagram showing a conic image.

FIG. 14 is a diagram illustrating a state in which an arm is searched for in an extracted thermal image.

FIG. 15 is a flowchart showing a processing procedure in an arm extraction step.

FIG. 16 is a schematic diagram showing how to calculate a cumulative temperature gradient.

FIG. 17 is a graph showing a relationship between a pixel value in an arm direction and a cumulative temperature gradient in each pixel when no heating is performed.

FIG. 18 is a graph showing the relationship between the pixel value in the arm direction and the cumulative temperature gradient in each pixel when heating at 0.6 ° C.

FIG. 19 is a graph showing the relationship between the pixel value in the arm direction and the cumulative temperature gradient in each pixel when heating at 1.2 ° C.

20 is a graph showing the relationship between the actual difference between the maximum cumulative temperature gradient T _x temperature difference and sound equipment and bad equipment sound gear and bad gear.

FIG. 21 is a perspective view showing a measurement object of Example 1.

FIG. 22 is a graph showing the results of Example 1 and Comparative Example.

FIG. 23 is a perspective view showing a measured object of Example 2.

FIG. 24 is a perspective view showing a measurement object of Example 3.

FIG. 25 is a graph showing the results of Example 3.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image extraction apparatus 2 Object 3 Thermal image 4 Visible image 5 Distance image 6 Distance image creation means 7 Corresponding point extraction means 8 Matrix creation means 9 Image matching means 10 Image extraction means 11 Infrared camera 12 Visible light camera 14 Distribution pole 15 pillar Upper equipment 16 Abnormality detector for pole equipment on distribution pole 17 Arm 18 Arm extraction means 19 Temperature gradient calculation means 20 Abnormality judgment means

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 1/00 315 G06T 1/00 315 5C054 7/60 180 7/60 180B 5L096 H04N 5/225 H04N 5 / 225 C 7/18 7/18 K F term (reference) 2F065 AA06 AA51 CC00 FF05 FF09 FF69 GG09 JJ03 JJ05 QQ00 QQ17 QQ21 QQ25 QQ32 QQ33 QQ38 QQ41 2F112 AC06 BA01 BA16 DA28 FA32 FA38 FA41 2G066 AC09 BA04 BA04 CA12 CA16 CB01 CB13 CB16 CE16 DA01 DA07 DB03 DB06 DC09 DC16 5C022 AA01 AA15 5C054 AA01 AA04 CA04 CA05 FC12 FC15 HA05 HA20 5L096 AA02 AA09 BA03 BA18 CA05 DA04 EA45 FA04 FA06 FA15 GA12 GA41 HA01

Claims (7)

[Claims] 1. A distance image creating step of creating a distance image in which a color is changed for each distance based on distance information obtained from a plurality of visible images of an object, and a thermal image of the object and a distance image of the distance image. A corresponding point extraction step of finding a corresponding point, a matrix creation step of finding a transformation matrix that matches the corresponding points of the distance image and the thermal image, and specifying the target object in the distance image and before the specific or An image matching step of converting one of the region including the object and the thermal image of the distance image using the conversion matrix to match the two images, and using the position of the object in the distance image after the matching An image extracting step of extracting the object from the thermal image. 2. In the image matching step, the distance image is converted by the conversion matrix to match the thermal image, and then the object is specified in the converted distance image. The image extraction method according to claim 1. 3. A distance image creating means for creating a distance image in which a color is changed for each distance based on distance information obtained from a plurality of visible images of an object, and a thermal image of the object and a distance image of the distance image. Corresponding point extracting means for finding corresponding points, matrix creating means for finding a transformation matrix that matches the corresponding points of the distance image and the thermal image, and specifying the object in the distance image and specifying the object before or Image matching means for converting one of the region including the object in the distance image and the thermal image using the conversion matrix to match the two images, and using the position of the object in the distance image after the matching Image extracting means for extracting the object from the thermal image. 4. A computer for extracting a thermal image of the object using the thermal image of the object and the plurality of visible images, based on distance information obtained from the plurality of visible images of the object. A distance image creating unit that creates a distance image with a different color for each, a corresponding point extracting unit that determines a corresponding point of the thermal image of the object and the corresponding distance image, and a corresponding point of the corresponding distance image and the thermal image. Matrix creation means for determining a conversion matrix to be matched, and specifying the object in the distance image and specifying one of the region including the object in the distance image and the thermal image before or after the identification using the conversion matrix. Image matching means for converting and matching the two images, and functioning as image extracting means for extracting the object from the thermal image using the position of the object in the distance image after the matching. An image extraction program characterized by causing 5. A power distribution pole having an arm and a pole-mounted device provided on the arm is photographed with one infrared camera to obtain a thermal image, and stereoscopically photographed with at least two visible light cameras to obtain each visible light. Obtaining one visible image at a time from an optical camera, processing the obtained thermal image and a plurality of the visible images by the image extraction method according to claim 1 or 2, and extracting the object from the thermal image. To create a thermal image after extraction, to extract the image of the arm from the thermal image after the extraction,
A method for detecting an abnormality of a pole equipment of a power distribution pole, comprising calculating a temperature gradient of the pole equipment and its surroundings based on the position of the arm to determine whether the pole equipment is normal or abnormal. 6. An infrared camera and at least two visible light cameras for photographing a power distribution pole including an arm and a pole-mounted device provided on the arm, a thermal image obtained by the infrared camera, and The image extraction device according to claim 3, wherein the object is extracted from the thermal image based on the plurality of visible images obtained by a visible light camera to create a thermal image after extraction, and the arm from the thermal image after extraction. Arm extracting means for extracting the image of the above, temperature gradient calculating means for calculating the temperature gradient on the pole equipment and its surroundings based on the position of the arm, and whether the pole equipment is normal or abnormal based on the temperature gradient An abnormality detecting device for a pole-mounted device of a distribution pole, comprising: an abnormality determining means for determining. 7. A power distribution pole having an arm and a pole-mounted device provided on the arm is photographed by one infrared camera to obtain a thermal image, and stereoscopic photographed by at least two visible light cameras to obtain a visible image. Obtaining a visible image one by one from an optical camera, and using the obtained thermal image and a plurality of the visible images, a computer to determine whether the pole equipment is normal or abnormal by using the plurality of visible images. A distance image creating means for creating a distance image in which a color is changed for each distance based on the distance information obtained from the image; a corresponding point extracting means for finding corresponding points of the thermal image and the distance image; Matrix creation means for finding a transformation matrix that matches the corresponding points of the image, and an area including the object in the distance image, specifying the object in the distance image before and after the identification; and Image matching means for converting one of the thermal images by the conversion matrix to match both images, and extracting the object from the thermal image using the position of the object in the distance image after the matching. Image extraction means for creating a post-extraction thermal image, arm extraction means for extracting the image of the arm from the post-extraction thermal image, and calculating the temperature gradient on the pole equipment and its surroundings based on the position of the arm A program for functioning as a temperature gradient calculating means and as an abnormality judging means for judging whether the equipment on the pole is normal or abnormal based on the temperature gradient, an abnormality detection program for the equipment on the pole of the distribution pole.