KR20230026047A - Apparatus for determining abnormal state of power distribution facility and method thereof - Google Patents

Abstract

The present invention relates to an apparatus for determining an abnormal state of a power distribution facility, the apparatus comprising: a camera unit for generating a real image and a thermal image when a power distribution facility is photographed; a display unit for outputting a real image and a thermal image; and a control unit for detecting power distribution facilities as objects in the real image, generating a bounding box for each detected object as a result value, searching a thermal image for an area corresponding to the bounding box of each object, extracting the area from the thermal image, analyzing a temperature distribution based on the extracted bounding box for each object based on the thermal image, and determining an abnormal state of each power distribution facility based on an analysis result of the temperature distribution based on the bounding box and an abnormal state determination reference table previously stored in a database.

Description

Apparatus and method for determining abnormal state of power distribution equipment {APPARATUS FOR DETERMINING ABNORMAL STATE OF POWER DISTRIBUTION FACILITY AND METHOD THEREOF}

The present invention relates to an apparatus and method for determining an abnormal state of a power distribution facility, and more particularly, to acquire the temperature of a heating part in the power distribution facility by merging a thermal image and an RGB image of the power distribution facility, and the power distribution facility The present invention relates to an apparatus and method for determining an abnormal state of a power distribution facility, which can determine an abnormal state of the power distribution facility through a statistical analysis based on a temperature of a heating part of a power distribution facility.

In general, thermal imaging technology is a technology that detects radiant energy on the surface of an object to be detected, converts it into temperature, and provides real-time images. Since it is possible to measure changes in the surface temperature of the object to be detected in a non-contact, non-destructive and real-time manner, the material It is used in various fields such as thermal property evaluation, deterioration diagnosis, defect inspection, medical diagnosis through body heat measurement, fire or gas leak monitoring, or security area intrusion monitoring.

A thermal imaging camera to which this thermal imaging technology is applied is a device that accepts infrared wavelengths emitted from the unique radiant energy of all objects and converts the output electrical signal into a still image or moving image that can be recognized by humans and outputs it. As a method, motors, large parts, electrical panels, etc. can be measured at once, and overheating or other dangerous parts can be easily found no matter how small they are. That is, it has the same effect as using thousands of infrared thermometers at the same time. An infrared thermometer can measure the temperature of only one point, whereas a thermal imaging camera can measure the temperature of the entire target object at the same time.

In addition, there are two types of infrared sensors that convert infrared wavelengths into electrical signals in a thermal imaging camera: a thermal type room temperature method and a photon type low temperature method. Among them, the low temperature method has excellent performance, but requires a liquid nitrogen cooler and is expensive. On the other hand, the room temperature method is a method of detecting radiant energy as resistance, current, or electromotive force change. Compared to the low temperature method, the performance is inferior, but it does not require a cooler and is inexpensive, so it is commonly used.

However, since the infrared sensor of the room temperature method has a lower resolution than the low temperature method, there is a problem in that the object discrimination performance is very insufficient. However, since a lot of cost is consumed to use a high-resolution thermal imaging camera, there is a problem of causing an economic burden.

In order to solve the above-described problem, the resolution can be improved by synthesizing a real image captured by a general camera (or a visual camera, an optical camera) and a thermal image captured by a thermal camera. For example, a compact device has been developed that integrates the functions of a thermal imaging camera and a visual imaging camera so that a user can take pictures while carrying the camera.

However, because the resolution of a normal camera (or a visible camera, an optical camera) and a thermal imaging camera are different from each other, and both cameras (normal camera and thermal imaging camera) cannot be installed in the same location (i.e., installed in a stereo camera format) Therefore, there is a problem in that a matching error occurs according to the distance during synthesis (fusion) of the real image and the thermal image according to the distance of the subject.

In addition, recently, a thermal imaging camera is being used to check the condition of distribution facilities. Visual inspection to classify facilities is essential, and even if the temperature of the heating part of the power distribution facility can be known, only the temperature change is monitored, and it is difficult to accurately determine the state thereof.

Therefore, there is a need for a technology capable of accurately determining the current state and possible abnormal state based on the acquired temperature of the heat generating portion of the power distribution facility and the acquired temperature of the heat generating portion of the power distribution facility.

The background art of the present invention is disclosed in Republic of Korea Patent Registration No. 10-2096576 (registered on March 27, 2020, image synthesis method and apparatus).

According to one aspect of the present invention, the present invention has been created to solve the above problems, and the temperature of the heating part in the distribution facility is obtained by merging a thermal image and an RGB image obtained by photographing the distribution facility, An object of the present invention is to provide an apparatus and method for determining an abnormal state of a power distribution facility, which can determine an abnormal state of the power distribution facility through a statistical analysis based on a temperature of a heating portion of the power distribution facility.

An apparatus for determining an abnormal state of a power distribution facility according to an aspect of the present invention includes a camera unit that generates a real image and a thermal image when photographing a power distribution facility; a display unit outputting the real image and the thermal image; and detecting power distribution facilities as objects in the real image, generating a bounding box for each detected object as a result value, finding and extracting a region corresponding to the bounding box of each object in the thermal image, and Based on the thermal image, the temperature distribution is analyzed based on the bounding box for each object extracted above, and based on the analysis result of the temperature distribution based on the bounding box and the abnormal state judgment standard table stored in the database, each power distribution facility It is characterized in that it includes; a control unit for performing an abnormal state determination for the.

In the present invention, the camera unit includes an optical camera generating the real image and a thermal imaging camera generating the thermal image, and the optical camera is a stereo camera.

In the present invention, the control unit outputs or synthesizes and outputs images captured by the camera unit to the display unit, and includes a bounding box for each object detected in the real image and a bounding box for each object. It is characterized in that the temperature information based on is additionally synthesized and outputted to the image.

In the present invention, in order to determine an abnormal state based on the analysis result of the temperature distribution based on the bounding box for each object, for communicating with an external server in which accumulated statistical information and an abnormal state determination criterion table are stored in advance in the form of a database Characterized in that it further includes a; communication unit.

In the present invention, the temperature information and abnormality based on the bounding box for each object as a result of inspecting the same power distribution facility every time the real image and the thermal image captured by the camera unit and the power distribution facility are inspected. Characterized in that it further includes; a database for accumulatively storing state determination results.

In the present invention, the control unit acquires a pair of real image and thermal image from the camera unit, detects power distribution facilities as objects in the real image, and bounding boxes for each object detected in the real image. is generated, the bounding box is projected as a bounding box of the thermal image, and distribution equipment as an object is extracted from the thermal image based on the bounding box projected on the thermal image, and the distribution equipment image for each object is generated. A foreground/background segmentation mask is estimated, and the maximum, average, and minimum temperatures are estimated as temperature information of the distribution equipment for each object using the foreground/background division mask, and the estimated temperature information of the distribution equipment for each object It is characterized in that the abnormal state is determined for each distribution facility based on the abnormal state determination standard table previously stored in the database.

)를 열화상 이미지의 바운딩 박스(

)로 사영하며,In the present invention, in order to project the bounding box of the real image to the bounding box of the thermal image, the control unit uses Equation 1 below to project the bounding box of the real image (

) to the bounding box of the thermal image (

), and

(Equation 1)

는 상기 실화상 이미지의 RGB 좌표계의 바운딩 박스,

는 광학 카메라의 내부 파라미터,

는 열화상 카메라의 내부 파라미터,

는 광학 카메라와 열화상 카메라간의 내부 파라미터,

는 배전설비에 대한 거리정보인 것을 특징으로 한다.here

Is the bounding box of the RGB coordinate system of the real image,

is an internal parameter of the optical camera,

is an internal parameter of the thermal imaging camera,

is an internal parameter between the optical camera and the thermal imaging camera,

It is characterized in that is distance information about the distribution facility.

In the present invention, the control unit performs the following math based on the bounding box projected on the thermal image in order to extract a power distribution facility, which is an object, from the thermal image based on the bounding box projected on the thermal image. Using Equation 2, the distribution facility image for each object is extracted,

(Equation 2)

는 열화상 이미지의 바운딩 박스,

는 지정된 이미지 추출 함수(

)를 이용하여 열화상 이미지에서 추출한 각 객체별 부분 이미지, 상기 함수(

)는 바운딩 박스(

)의 좌표를 기준으로 이미지를 잘라내는 함수로서 열화상 이미지(

)에서 열화상 이미지의 바운딩 박스(

)의 부분 이미지를 취득하는 함수인 것을 특징으로 한다.here

is the bounding box of the thermal image,

is the specified image extraction function (

), the partial image for each object extracted from the thermal image, the function (

) is the bounding box (

) as a function of cropping the image based on the coordinates of the thermal image (

) in the bounding box of the thermal image (

) It is characterized in that it is a function for obtaining a partial image of.

)을 아래의 수학식 3을 이용하여 추정하되,In the present invention, the control unit, in order to estimate a foreground/background segmentation mask of the power distribution facility image for each object, an optimal foreground separation threshold of a power distribution facility image for each object extracted from a thermal image (

) is estimated using Equation 3 below,

(Equation 3)

)에 대한 전경/후경을 분리하기 위해 Binary OSTU(OMSK STATE TRANSPORT UNIVERSITY) 기반의 기 지정된 최적 전경 분리 임계값 추정 함수(

)를 적용한 임계값 알고리즘을 이용하는 것을 특징으로 한다.Partial image for each object (

), a predetermined optimal foreground separation threshold estimation function based on Binary OSTU (OMSK STATE TRANSPORT UNIVERSITY) to separate foreground/background for ) (

) It is characterized by using a threshold algorithm applied.

)를 아래의 수학식 4를 이용하여 추정하되,In the present invention, the control unit, in order to estimate the maximum, average, and minimum temperatures as temperature information of the power distribution equipment for each object using the foreground/background segmentation mask, the average temperature of the power distribution equipment for each object (

) is estimated using Equation 4 below,

(Equation 4)

)는 전경 분리를 위해 산출된 최적 전경 분리 임계값(

)을 적용한 객체의 온도를 의미하는 것을 특징으로 한다.where the temperature of each object (

) is the optimal foreground separation threshold calculated for foreground separation (

) is applied, characterized in that it means the temperature of the object.

According to another aspect of the present invention, a method for determining an abnormal state of a power distribution facility includes acquiring a pair of real image and thermal image from a camera unit by a control unit of an apparatus for determining an abnormal state of a power distribution facility; detecting, by the control unit, power distribution facilities as objects in the real image; generating, by the control unit, a bounding box for each object detected in the real image, and projecting the bounding box as a bounding box of the thermal image; extracting, by the control unit, a power distribution facility as an object from the thermal image based on a bounding box projected on the thermal image, and estimating a foreground/background division mask of the power distribution facility image for each object; estimating, by the control unit, maximum, average, and minimum temperatures as temperature information of a power distribution facility for each object using the foreground/background segmentation mask; and determining, by the control unit, the abnormal state of each power distribution facility based on the temperature information estimated from the power distribution equipment for each object and the abnormal state determination standard table pre-stored in the database.

)를 열화상 이미지의 바운딩 박스(

)로 사영하며,In the present invention, in the step of projecting the bounding box of the real image to the bounding box of the thermal image, the control unit uses Equation 1 below to determine the bounding box of the real image (

) to the bounding box of the thermal image (

), and

(Equation 1)

는 상기 실화상 이미지의 RGB 좌표계의 바운딩 박스,

는 광학 카메라의 내부 파라미터,

는 열화상 카메라의 내부 파라미터,

는 광학 카메라와 열화상 카메라간의 내부 파라미터,

는 배전설비에 대한 거리정보인 것을 특징으로 한다.here

Is the bounding box of the RGB coordinate system of the real image,

is an internal parameter of the optical camera,

is an internal parameter of the thermal imaging camera,

is an internal parameter between the optical camera and the thermal imaging camera,

It is characterized in that is distance information about the distribution facility.

In the present invention, in the step of extracting a power distribution facility, which is an object, from the thermal image based on the bounding box projected on the thermal image, the control unit performs the following based on the bounding box projected on the thermal image Using Equation 2, an image of a distribution facility for each object is extracted,

(Equation 2)

는 열화상 이미지의 바운딩 박스,

는 지정된 이미지 추출 함수(

)를 이용하여 열화상 이미지에서 추출한 각 객체별 부분 이미지, 상기 함수(

)는 바운딩 박스(

)의 좌표를 기준으로 이미지를 잘라내는 함수로서 열화상 이미지(

)에서 열화상 이미지의 바운딩 박스(

)의 부분 이미지를 취득하는 함수인 것을 특징으로 한다.here

is the bounding box of the thermal image,

is the specified image extraction function (

), the partial image for each object extracted from the thermal image, the function (

) is the bounding box (

) as a function of cropping the image based on the coordinates of the thermal image (

) in the bounding box of the thermal image (

) It is characterized in that it is a function for obtaining a partial image of.

)을 아래의 수학식 3을 이용하여 추정하되,In the present invention, in the step of estimating the foreground/background segmentation mask of the power distribution facility image for each object, the controller includes an optimal foreground separation threshold of the power distribution facility image for each object extracted from the thermal image (

) is estimated using Equation 3 below,

(Equation 3)

)에 대한 전경/후경을 분리하기 위해 Binary OSTU(OMSK STATE TRANSPORT UNIVERSITY) 기반의 기 지정된 최적 전경 분리 임계값 추정 함수(

)를 적용한 임계값 알고리즘을 이용하는 것을 특징으로 한다.Partial image for each object (

), a predetermined optimal foreground separation threshold estimation function based on Binary OSTU (OMSK STATE TRANSPORT UNIVERSITY) to separate foreground/background for ) (

) It is characterized by using a threshold algorithm applied.

)를 아래의 수학식 4를 이용하여 추정하되,In the present invention, in the step of estimating the maximum, average, and minimum temperatures as temperature information of the power distribution equipment for each object using the foreground/background segmentation mask, the control unit includes the average temperature of the power distribution equipment for each object (

) is estimated using Equation 4 below,

(Equation 4)

)는 전경 분리를 위해 산출된 최적 전경 분리 임계값(

)을 적용한 객체의 온도를 의미하는 것을 특징으로 한다.where the temperature of each object (

) is the optimal foreground separation threshold calculated for foreground separation (

) is applied, characterized in that it means the temperature of the object.

According to one aspect of the present invention, the present invention acquires the temperature of the heating part in the distribution facility by merging the RGB image with the thermal image of the distribution facility, and statistics based on the temperature of the heating part of the distribution facility. It is possible to determine the abnormal state of the distribution facility through a systematic analysis.

1 is an exemplary view showing a schematic configuration of a device for determining an abnormal state of a power distribution facility according to an embodiment of the present invention.
2 is a flowchart illustrating a method for determining an abnormal state of a power distribution facility according to an embodiment of the present invention.
FIG. 3 is an exemplary view showing a method of detecting temperature by setting a bounding box for each power distribution facility using a real image and a thermal image in FIG. 2;
FIG. 4 is an exemplary view showing an abnormal state judgment standard table according to temperature for each distribution facility in FIG. 2;
FIG. 5 is an exemplary view showing a real image and a thermal image acquired through a camera unit in FIG. 2;
6 is an exemplary view showing a foreground/background segmentation mask of an image for each object in FIG. 2;
FIG. 7 is a view of FIG. 2 in which only foreground/background separation is performed in the temperature estimation algorithm according to the present embodiment (a), foreground/boundary/background separation is performed (b), and additional morphological calculations are performed. An example showing the resulting image (c).
FIG. 8 is an exemplary view showing images for each temperature divided through the OSTU algorithm in FIG. 2;
FIG. 9 is an exemplary diagram showing temperature estimation results for each object based on a thermal image in FIG. 2;

Hereinafter, an embodiment of an apparatus and method for determining an abnormal state of a power distribution facility according to the present invention will be described with reference to the accompanying drawings.

In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of description. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, definitions of these terms will have to be made based on the content throughout this specification.

1 is an exemplary diagram showing a schematic configuration of a device for determining an abnormal state of a power distribution facility according to an embodiment of the present invention.

As shown in FIG. 1, the apparatus for determining the abnormal state of a power distribution facility according to this embodiment includes a camera unit 110, a sensor unit 120, a control unit 130, a display unit 140, a communication unit 150, and a database 160 (or storage unit).

The camera unit 110 includes an optical camera and a thermal imaging camera.

The optical camera creates a real image (RGB-image) by photographing a thermally-sensing object, and uses a CCD image sensor like a CCD (Charge Coupled Device) camera or a CMOS image like a Complementary Metal Oxide Semiconductor (CMOS) camera. sensor can be used.

The thermal imaging camera creates a thermal image by photographing a thermally-sensing target.

The optical camera and the thermal imaging camera are separated by a specified distance.

The sensor unit 120 may include a distance sensor and a temperature sensor.

The distance sensor measures a distance to a thermal sensing target, and the temperature sensor measures a temperature around the thermal sensing target.

The information detected through the sensor unit 120 may be used to supplement the information detected through the camera unit 110 (e.g., internal/external temperature of the power distribution facility), and also more accurately monitor the power distribution facility. It can also be used to differentiate.

The control unit 130 outputs a preview image (ie, a real image, an RGB-image) of a thermal sensing target photographed by an optical camera of the camera unit 110 to the display unit 140, and outputs the thermal image A thermal image of the object to be detected by the camera is output to the display unit 140 .

The control unit 130 may synthesize the real image (RGB-image) and the thermal image and output the synthesized image to the display unit 140 .

In addition, the controller 130 stores a preview image (ie, a real image, an RGB-image) of a thermal detection target captured by the optical camera and a thermal image of the thermal detection target captured by the thermal imaging camera into the database ( 160) to save.

The control unit 130 performs object detection on the electric pole equipment in the real image, obtains a bounding box for each detected object as a result value, and finds and extracts an area corresponding to each detected object from the thermal image. .

In addition, the control unit 130 analyzes the temperature distribution of the inside (inside) area and the outside (outside) area of the bounding box for each object with respect to the bounding box of the extracted area (the area corresponding to each object), and the bounding box for each object When an abnormally high temperature in the inner region compared to the outer region of the is detected, an abnormal state is determined (see FIG. 3).

The display unit 140 outputs a video (image) taken by the camera unit 110, and information related to the video (image) added by the controller 140 (eg, bounding box, inside/outside of bounding box) external temperature information, etc.) is output.

The communication unit 150 uses an external server (not shown) in which statistical information of corresponding information (i.e., an abnormal state according to internal/external temperature for each distribution facility) is stored in advance in order to determine an abnormal state according to internal/external temperature for each object. city) communicate with

The database 160 stores a real image captured by the camera unit 110 and a thermal image corresponding thereto. In addition, whenever a distribution facility is inspected, inspection results (eg, internal/external temperature of the distribution facility, abnormal state determination result, etc.) of the same distribution facility are cumulatively stored in the database 160 . Through this, it is possible to continuously monitor the status change of the distribution facility in the form of a chart.

Hereinafter, the operation of the controller 130 will be described in more detail.

In addition, in the present embodiment, the power distribution facility includes facilities (eg, transformers, etc.) (or utility pole facilities) installed on poles (or overhead poles).

FIG. 2 is a flowchart illustrating a method for determining an abnormal state of a power distribution facility according to an embodiment of the present invention, and FIG. 3 is a flowchart for setting a bounding box for each power distribution facility using a real image and a thermal image in FIG. 2 . It is an example diagram showing how to detect the temperature by

Referring to FIG. 2 , the controller 130 acquires a pair of images (a real image and a thermal image) from the camera unit 110 (S101).

In addition, the control unit 130 detects the power distribution facility in the real image (or RGB image) using the designated power distribution facility detection algorithm (S102).

In addition, the controller 130 creates a bounding box in the power distribution equipment detected in the real image (or RGB image), acquires average distance information of electric poles (processed electric poles) (S103) (eg, stereo Acquisition of distance information using depth using a camera), and the bounding box of the real image is projected as a bounding box of a thermal image camera coordinate system (ie, a thermal image) (S104).

In addition, the control unit 130 crops object images (ie, power distribution facilities in the thermal image) based on the bounding box of the thermal image (S105), and images for each object (ie, each power distribution facility) A foreground/background segmentation mask is estimated (S106).

In addition, the control unit 130 estimates the maximum, average, and minimum temperatures for each object (ie, each distribution facility) using the foreground/background segmentation mask estimated from the image for each object (ie, each distribution facility). Do (S107).

In addition, the control unit 130 performs a rule-based abnormal state determination based on the estimated temperature for each object (ie, each power distribution facility) (S108).

Hereinafter, each step (S101 to S108) will be described in more detail.

The step of projecting the bounding box of the real image into the bounding box of the thermal camera coordinate system (S104) (that is, a fusion method of the real image and the thermal image) will be described in detail.

The controller 130 converts the bounding box of the optical camera coordinate system (or RGB coordinate system) into the thermal camera coordinate system (or thermal coordinate system) using Equation 1 below.

)를 카메라 캘리브레이션 과정에서 얻어진 광학 카메라(즉, RGB 카메라)의 내부 파라미터(intrinsic parameter)(

)와 열화상 카메라(즉, Thermal 카메라)의 내부 파라미터(

)와 상기 두 카메라(즉, RGB-Thermal 카메라)간의 내부 파라미터(

)를 이용하여 좌표계 변환을 수행한다. 이 때 좌표계 변환 과정에서 전주(배전설비)에 대한 거리정보(

)가 필요한데, 해당 전주(배전설비)에 대한 거리정보는 상기 카메라부(110)(즉, 스테레오 카메라)로부터 취득되는 전주(배전설비)별 거치추정 결과를 활용한다. 이러한 좌표 변환 과정을 통해 실화상 이미지(즉, RGB 이미지)에서 검출된 바운딩 박스를 열화상 이미지(즉, Thermal 이미지)에 투영이 가능하다. Here, the bounding box of the RGB coordinate system (

) is the intrinsic parameter of the optical camera (i.e., RGB camera) obtained in the camera calibration process (

) and the internal parameters of the thermal imaging camera (i.e., Thermal camera) (

) and the internal parameters between the two cameras (i.e., RGB-Thermal camera) (

) to perform coordinate system transformation. At this time, in the process of converting the coordinate system, distance information (

) is required, and the distance information for the corresponding pole (distribution facility) uses the distance estimation result for each pole (distribution facility) acquired from the camera unit 110 (ie, stereo camera). Through this coordinate conversion process, it is possible to project a bounding box detected in a real image (ie, an RGB image) onto a thermal image (ie, a thermal image).

For reference, simple movement of the bounding box without the coordinate conversion process is not applicable because the two cameras (ie, RGB-Thermal cameras) have different viewing angles, resolutions, and sensor sizes.

However, there is a possibility that it may not be accurately projected due to the inaccuracy of the parameters between the two cameras and the inaccuracy of the estimated pole (distribution facility) distance during the coordinate conversion process. You can adjust the size of the bounding box.

The step of cropping (S105) the object image (ie, the power distribution equipment in the thermal image) by the control unit 130 based on the bounding box of the thermal image will be described in more detail.

Based on the bounding box converted to the thermal camera coordinate system (or thermal coordinate system) through the coordinate conversion, an image for each object may be extracted (cropped) using Equation 2 below.

)를 지정된 함수(

)를 이용하여 열화상 이미지를 각 객체별 이미지(

)로 추출한다(crop)(또는 자른다). 여기서 상기 지정된 함수(

)는 바운딩 박스(

)의 좌표를 기준으로 이미지를 잘라내는 함수로 열화상 이미지(

)에서

의 부분 이미지를 취득한다.Here, the bounding box of the thermal camera coordinate system (or thermal coordinate system) (

) to the specified function (

) to convert the thermal image into an image for each object (

) to extract (crop) (or cut). Here, the above-specified function (

) is the bounding box (

) as a function that cuts out the image based on the coordinates of the thermal image (

)at

Acquire a partial image of

The step of estimating, by the control unit 130, the foreground/background division mask of the cropped image for each object (that is, for each power distribution facility) (S106) will be described in more detail.

)을 아래의 수학식 3을 이용하여 추정한다.The control unit 130 determines the optimal foreground separation threshold of the cropped image (

) is estimated using Equation 3 below.

)에 대한 전경/후경을 분리하기 위해 Binary OSTU(OMSK STATE TRANSPORT UNIVERSITY) 기반의 최적 전경 분리 임계값 추정 함수(

)를 적용한 임계값 알고리즘을 이용할 수 있다.Here, the image for each object (

Optimal foreground separation threshold estimation function based on Binary OSTU (OMSK STATE TRANSPORT UNIVERSITY) to separate foreground/background for ) (

) can be used.

For example, the binary OSTU-based initial foreground and background are separated, and then the object border and the refined foreground are separated by performing a Binary OSTU-based threshold algorithm again on the separated initial foreground. In the case of the object edge, separation is necessary because the temperature value is inaccurate due to mixing with the ambient temperature. Finally, an optimal separation threshold is estimated by performing a morphology close operation to estimate only the internal temperature.

In particular, since a temperature blur occurs for objects in a thermal image in a driving environment, and an inaccurate temperature boundary between foreground and background is observed, foreground/background segmentation masks of images for each object (ie, each power distribution facility) There is an effect of enabling more accurate temperature estimation through the estimation process.

The control unit 130 estimates maximum, average, and minimum temperatures for each object (ie, each distribution facility) using a foreground/background segmentation mask estimated from an image for each object (ie, each distribution facility) Step S107 will be described in more detail.

)는 다음 수학식 4를 이용하여 추정할 수 있다.The average temperature of each object (ie, each power distribution facility) (

) can be estimated using Equation 4 below.

)는 전경 분리를 위해 산출된 최적 전경 분리 임계값(

)을 적용한 객체의 온도를 의미하며, 여기에 픽셀 레벨(pixel-level) 온도 값 평균을 계산하여 각 객체별 평균온도를 산출한다. 따라서 한 이미지에서의 객체 평균 온도(상간 평균 온도)는 상기 수학식 4와 같은 객체별 평균 온도를 기준으로 추정하게 된다. where the temperature of each object (

) is the optimal foreground separation threshold calculated for foreground separation (

) is applied, and the average temperature of each object is calculated by calculating the average of the pixel-level temperature values. Therefore, the average object temperature (average temperature between phases) in one image is estimated based on the average temperature of each object as shown in Equation 4 above.

The step (S108) of the control unit 130 performing a rule-based abnormal state determination based on the estimated temperature for each object (ie, each power distribution facility) will be described in more detail.

)를 (객체 최대온도 - 객체 평균온도의 절대값)(

)을 이용하여 계산하고, 해당 온도차와 미리 지정된 이상상태 판단 기준표(도 4 참조)를 활용하여 이상상태 여부를 판단한다.As shown in Equation 5 above, the temperature difference of the object (

) (maximum object temperature - absolute value of object average temperature) (

) is used to calculate, and the abnormal state is determined by utilizing the corresponding temperature difference and a predetermined abnormal state judgment standard table (see FIG. 4).

That is, the control unit 130 determines an abnormal state by using the temperature inferred from the thermal image. In the power distribution facility, it is difficult to judge based on the temperature value because the standards for each object, state, and temperature difference are different for each diagram. It is necessary to determine one clear criterion, and it is possible to determine whether an abnormal state exists by setting an abnormal state judgment standard table (see FIG. 4) according to the standards of the American Electrical Testing Association (NETA) and the thermal imaging diagnosis criteria for each power distribution facility.

At this time, the abnormal state determination criterion table (see FIG. 4) may be stored in the internal database 160 or stored in an external server (not shown) and transferred to the external server (not shown) through the communication unit 150. It is possible to determine the abnormal state by accessing and referring to the abnormal state determination criteria table (see FIG. 4).

FIG. 4 is an exemplary view showing an abnormal state judgment standard table according to temperature for each power distribution facility in FIG. 2 .

The abnormal state determination result is output through the display unit 140 .

Hereinafter, the drawings shown in FIGS. 5 to 9 will be described.

FIG. 5 is an exemplary view showing a real image and a thermal image obtained through the camera unit in FIG. 2. If a power distribution facility in a temperature rising state exists, a high temperature area is measured and a bright pixel (white) is displayed in the thermal image. is measured as

This means that it is difficult to distinguish each power distribution facility by image recognition. In particular, when power distribution facilities are overlapped, it is difficult to specify the boundary. Therefore, a real image of the same location is simultaneously acquired and fused with a thermal image (Fusion). ), it is possible to detect the distribution facility, and the temperature distribution for each distribution facility can be comprehensively acquired.

Accordingly, in this embodiment, a visual-thermal image fusion process is performed on an image of a processed power distribution facility (eg, suspension insulator, COS, polymer insulator, line post insulator, etc.) using an optical camera and a thermal imaging camera. At this time, in image processing acquired in the driving environment of a vehicle equipped with a camera for actually inspecting power distribution facilities, a temperature blur occurs on the thermal image due to the vehicle speed and the temperature of the object, and a bounding box detected in the real image. When is projected onto a thermal image, it is possible to classify the temperature range and obtain an appropriate temperature measurement value.

6 is an exemplary diagram showing foreground/background segmentation masks of images for each object in FIG. 2, wherein (a) is a foreground segmentation mask, (b) is an original thermal image image, and (c) is an abnormal state segmentation mask. is an example showing

FIG. 7 is a view of FIG. 2 in which only foreground/background separation is performed in the temperature estimation algorithm according to the present embodiment (a), foreground/boundary/background separation is performed (b), and additional morphological calculations are performed. It is an exemplary view showing the resulting image (c).

As shown in this, in the case of temperature estimation of an insulator in the initial temperature estimation experiment, many errors are confirmed. In particular, errors may occur due to mixing of foreground object and background temperature at the interface. As a complement to this, the foreground object edge is removed, and an improved temperature estimation result can be calculated as in the center image. Also, when the edge temperature value is continuously included, a foreground mask of a reliable part is obtained according to morphology operation This allows for more accurate temperature estimation.

FIG. 8 is an exemplary diagram showing images for each temperature divided through the OSTU algorithm in FIG. 2 , and FIG. 9 is an example diagram showing a temperature estimation result for each object based on a thermal image in FIG. 2 .

As described above, this embodiment improves efficiency by enabling automatic judgment of abnormal conditions at the same time as capturing real images and thermal images of power distribution facilities, temperature acquisition according to facility recognition, classification of facilities (COS, transformers, etc.), facilities It is possible to distinguish location/area, so it is possible to apply automation and advanced diagnosis technology.

The present invention has been described above with reference to the embodiments shown in the drawings, but this is only exemplary, and those skilled in the art can make various modifications and equivalent other embodiments. you will understand the point. Therefore, the technical protection scope of the present invention should be determined by the claims below. Implementations described herein may also be embodied in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (eg, discussed only as a method), the implementation of features discussed may also be implemented in other forms (eg, an apparatus or program). The device may be implemented in suitable hardware, software and firmware. The method may be implemented in an apparatus such as a processor, which is generally referred to as a processing device including, for example, a computer, microprocessor, integrated circuit, programmable logic device, or the like. Processors also include communication devices such as computers, cell phones, personal digital assistants ("PDAs") and other devices that facilitate communication of information between end-users.

110: camera unit 120: sensor unit
130: control unit 140: display unit
150: communication unit 160: database

Claims (15)

A camera unit that generates a real image and a thermal image when photographing a power distribution facility;
a display unit outputting the real image and the thermal image; and
Power distribution facilities are detected as objects in the real image, a bounding box for each detected object is generated as a result value, an area corresponding to the bounding box of each object is found and extracted from the thermal image, and the heat Based on the video image, the temperature distribution is analyzed based on the bounding box for each object extracted above, and based on the analysis result of the temperature distribution based on the bounding box and the abnormal state judgment standard table previously stored in the database, for each power distribution facility An abnormal state determination device for a power distribution system comprising a control unit for determining an abnormal state.
The method of claim 1, wherein the camera unit,
An optical camera generating the real image and a thermal imaging camera generating the thermal image,
The optical camera is a stereo camera.
The method of claim 1, wherein the control unit,
Outputting or synthesizing images captured by the camera unit to the display unit respectively,
The apparatus for determining abnormal conditions in power distribution facilities, characterized in that the video is further synthesized with a bounding box for each object detected in the real image and temperature information based on the bounding box for each object, and then outputted.
According to claim 1,
Further included is a communication unit for communicating with an external server in which accumulated statistical information and an abnormal state determination criterion table are stored in advance in the form of a database in order to determine an abnormal state based on the analysis result of the temperature distribution based on the bounding box for each object. A device for determining an abnormal state of a power distribution facility, characterized in that for doing.
According to claim 1,
A real image and a thermal image captured by the camera unit, and
A database for accumulatively storing temperature information based on the bounding box for each object and abnormal state determination results as inspection results for the same distribution facility whenever the power distribution facility is inspected. Abnormal state judgment device.
The method of claim 1, wherein the control unit,
Acquire a pair of real image and thermal image from the camera unit, detect power distribution facilities as objects in the real image, create a bounding box for each object detected in the real image, and open the bounding box It is projected as a bounding box of the visual image, and based on the bounding box projected on the thermal image, a distribution facility, which is an object, is extracted from the thermal image, and a foreground / background division mask of the distribution facility image is estimated for each object, Maximum, average, and minimum temperatures are estimated as the temperature information of the distribution equipment for each object using the foreground/background segmentation mask, and the temperature information estimated from the distribution equipment for each object and the abnormal state judgment standard table pre-stored in the database An abnormal state determination device of a distribution facility, characterized in that for performing an abnormal state determination for each distribution facility based on.
The method of claim 6, wherein the control unit,
In order to project the bounding box of the real image to the bounding box of the thermal image, using Equation 1 below, the bounding box of the real image (

) to the bounding box of the thermal image (

), and
(Equation 1)

here

Is the bounding box of the RGB coordinate system of the real image,

is an internal parameter of the optical camera,

is an internal parameter of the thermal imaging camera,

is an internal parameter between the optical camera and the thermal imaging camera,

Is the distance information of the distribution facility.
The method of claim 6, wherein the control unit,
In order to extract a power distribution facility as an object from the thermal image based on the bounding box projected on the thermal image, Equation 2 below is used based on the bounding box projected on the thermal image, and power distribution equipment for each object is used. extract the image,
(Equation 2)

here

is the bounding box of the thermal image,

is the specified image extraction function (

), the partial image for each object extracted from the thermal image, the function (

) is the bounding box (

) as a function of cropping the image based on the coordinates of the thermal image (

) in the bounding box of the thermal image (

) is a function for acquiring a partial image of a power distribution facility, characterized in that the device for determining the abnormal state.
The method of claim 6, wherein the control unit,
In order to estimate the foreground/background division mask of the distribution equipment image for each object, the optimal foreground separation threshold of the distribution equipment image for each object extracted from the thermal image (

) is estimated using Equation 3 below,
(Equation 3)

Partial image for each object (

), a predetermined optimal foreground separation threshold estimation function based on Binary OSTU (OMSK STATE TRANSPORT UNIVERSITY) to separate foreground/background for ) (

) Apparatus for determining the abnormal state of a power distribution facility, characterized in that using a threshold algorithm applied.
The method of claim 6, wherein the control unit,
In order to estimate the maximum, average, and minimum temperatures as temperature information of the distribution equipment for each object using the foreground / background segmentation mask,
Average temperature of distribution facilities for each object (

) is estimated using Equation 4 below,
(Equation 4)

where the temperature of each object (

) is the optimal foreground separation threshold calculated for foreground separation (

) An abnormal state determination device for distribution equipment, characterized in that it means the temperature of the object to which ) is applied.
acquiring a pair of real image and thermal image from a camera unit by a control unit of an abnormal state determination device of a power distribution facility;
detecting, by the control unit, power distribution facilities as objects in the real image;
generating, by the control unit, a bounding box for each object detected in the real image, and projecting the bounding box as a bounding box of the thermal image;
extracting, by the control unit, a power distribution facility as an object from the thermal image based on a bounding box projected on the thermal image, and estimating a foreground/background division mask of the power distribution facility image for each object;
estimating, by the control unit, maximum, average, and minimum temperatures as temperature information of a power distribution facility for each object using the foreground/background segmentation mask; and
The control unit determining an abnormal state for each distribution facility based on the temperature information estimated from the distribution facility for each object and an abnormal state determination standard table pre-stored in the database; How to judge status.
12. The method of claim 11, wherein in the step of projecting a bounding box of the real image to a bounding box of the thermal image,
The control unit,
Using Equation 1 below, the bounding box of the real image (

) to the bounding box of the thermal image (

), and
(Equation 1)

here

Is the bounding box of the RGB coordinate system of the real image,

is an internal parameter of the optical camera,

is an internal parameter of the thermal imaging camera,

is an internal parameter between the optical camera and the thermal imaging camera,

Is the distance information for the distribution facility.
The method of claim 11 , wherein in the step of extracting a power distribution facility as an object from the thermal image based on a bounding box projected on the thermal image,
The control unit,
Based on the bounding box projected on the thermal image, Equation 2 below is used to extract a distribution facility image for each object,
(Equation 2)

here

is the bounding box of the thermal image,

is the specified image extraction function (

), the partial image for each object extracted from the thermal image, the function (

) is the bounding box (

) as a function of cropping the image based on the coordinates of the thermal image (

) in the bounding box of the thermal image (

).
The method of claim 11, wherein in the step of estimating a foreground/background segmentation mask of the power distribution facility image for each object,
The control unit,
The optimal foreground separation threshold of the distribution facility image for each object extracted from the thermal image (

) is estimated using Equation 3 below,
(Equation 3)

Partial image for each object (

), a predetermined optimal foreground separation threshold estimation function based on Binary OSTU (OMSK STATE TRANSPORT UNIVERSITY) to separate foreground/background for ) (

A method for determining an abnormal state of a power distribution facility, characterized in that using a threshold algorithm to which ) is applied.
12. The method of claim 11, wherein in the step of estimating maximum, average, and minimum temperatures as temperature information of the power distribution equipment for each object using the foreground/background segmentation mask,
The control unit,
Average temperature of distribution facilities for each object (

) is estimated using Equation 4 below,
(Equation 4)

where the temperature of each object (

) is the optimal foreground separation threshold calculated for foreground separation (

) A method for determining the abnormal state of a power distribution facility, characterized in that it means the temperature of the object to which ) is applied.

Images