CN110942458B - Temperature anomaly defect detection and positioning method and system - Google Patents
Temperature anomaly defect detection and positioning method and system Download PDFInfo
- Publication number
- CN110942458B CN110942458B CN201911244645.9A CN201911244645A CN110942458B CN 110942458 B CN110942458 B CN 110942458B CN 201911244645 A CN201911244645 A CN 201911244645A CN 110942458 B CN110942458 B CN 110942458B
- Authority
- CN
- China
- Prior art keywords
- image
- visible light
- module
- temperature
- positioning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000007547 defect Effects 0.000 title claims abstract description 34
- 230000005856 abnormality Effects 0.000 claims abstract description 33
- 238000010191 image analysis Methods 0.000 claims abstract description 28
- 238000003384 imaging method Methods 0.000 claims abstract description 27
- 238000001931 thermography Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 23
- 230000009466 transformation Effects 0.000 claims description 32
- 230000002159 abnormal effect Effects 0.000 claims description 21
- 238000013519 translation Methods 0.000 claims description 18
- 238000000354 decomposition reaction Methods 0.000 claims description 15
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 12
- 230000003993 interaction Effects 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 9
- 230000000877 morphologic effect Effects 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 238000010845 search algorithm Methods 0.000 claims description 5
- 230000004807 localization Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000009529 body temperature measurement Methods 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241001270131 Agaricus moelleri Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000008719 thickening Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20036—Morphological image processing
Abstract
The invention relates to a temperature anomaly defect detection and positioning method and a system, wherein the system comprises the following steps: the system comprises an infrared thermal imaging temperature measuring module, a visible light imaging module, a parameter setting module, a result display module, an alarm module and an image analysis positioning module; the method comprises the following steps: s1: fixing the relative positions of the infrared thermal imaging temperature measuring module and the visible light imaging module, wherein the mirror surfaces of the infrared thermal imaging temperature measuring module and the visible light imaging module are relatively parallel; the infrared thermal imaging temperature measuring module collects infrared thermal image data, and the visible light imaging module collects visible light image data; transmitting the data of the two to an image analysis positioning module; s2: the image analysis positioning module receives infrared heat map data and visible light image data, and temperature abnormality detection and positioning results are obtained through image analysis processing; s3: and transmitting the image and text information of the temperature abnormality detection positioning result to a result display module for display, and transmitting the temperature abnormality detection signal to an alarm module for alarm. The invention can improve the efficiency of judging and positioning the temperature anomaly defect.
Description
Technical Field
The invention relates to the field of detection and positioning of temperature anomaly defects, in particular to a detection and positioning method and a detection and positioning system of temperature anomaly defects.
Background
And detecting and positioning the temperature anomaly defect by using an image, and acquiring temperature data by using a thermal infrared imager to acquire an infrared temperature heat map so as to position the temperature anomaly defect. But most thermal infrared imagers have lower resolution and far lower resolution than common visible light cameras, which results in serious loss of infrared heat map details and blurred images. At present, the location of the temperature anomaly defect often determines a temperature anomaly region approximately according to the shooting position of the inspection robot or the shooting position of a technician, and a worker familiar with the site and an infrared heat map needs to remotely check or enter the site to further check the temperature anomaly region to determine the specific position of the temperature anomaly. Therefore, the working load is large, the efficiency is low, and meanwhile, the workers are required to have better professional skills and rich practical experience.
Chinese patent CN 110223330A proposes a method and system for registering visible and infrared images. The patent collects visible light images and infrared images of the power equipment; performing primary registration on the infrared image and the visible light image; three pairs of the initial registration point pairs are arbitrarily selected to obtain an affine transformation matrix set, a set with highest image edge correlation degree in the affine transformation matrix set is obtained, so that three optimal registration point pairs are obtained, and a registration point neighborhood is constructed; and traversing and searching the sub-pixel points with the highest affine edge correlation degree in the registration point neighborhood by using the selected sub-pixel units, and finally obtaining the registration point pair. The patent uses corner detection to extract features, and introduces edge correlation to calculate registration point pairs. The feature point pairing is complex and complicated, and large deviation is easy to occur in image registration due to locally obvious features.
Chinese patent CN 110335271A proposes an infrared detection method and device for electrical component faults. The patent collects infrared and visible light images of normal electrical components as standard images, and collects infrared and visible light images of electrical components to be detected during detection; the infrared images and the visible light images of the normal component and the electrical component to be tested are correspondingly processed to realize the complete matching of the visible light images and the infrared images; and carrying out differential comparison analysis on gray images obtained by carrying out gray processing on the infrared images of the two completely matched to-be-detected and normal electrical components so as to determine the abnormal condition of the to-be-detected electrical components. The patent realizes that the complete matching of the visible light image and the infrared image can be realized through calibration, when the relative positions of the infrared thermal imager and the visible light camera deviate due to daily use, the matching is deviated, and the complete matching can be normally completed only through recalibration or adjustment of the relative positions of the cameras.
At present, the temperature anomaly defect detection and positioning mainly has the following defects:
(1) The difficulty of feature point registration is high. Due to the difference of imaging modes and imaging platforms among the sensors, even the infrared image and the visible light image which are obtained by imaging the same area at the same time have large difference. The infrared image and the visible light image have great differences in resolution and color, so that the registration of the infrared image and the visible light image is difficult to realize; in terms of resolution, infrared images are mostly much lower than the resolution of common visible light images, which results in serious loss of grayscale details of the infrared images, blurred images, and large differences from sharp textures in the visible light grayscale images. Feature point extraction can be realized faster, but high-precision feature point pairing is realized, complex descriptor establishment is often involved, and the influence caused by size, resolution, direction and the like is solved.
(2) Camera position requirements are high. The camera calibration mode is adopted to realize the registration of the visible light image and the infrared image more quickly and simply. However, when the relative positions of the thermal infrared imager and the visible light camera deviate due to the fact that the relative positions of the cameras deviate due to normal vibration and the like in daily use, deviation occurs in position mapping, and the relative positions of the cameras must be recalibrated or adjusted to normally finish image registration.
Disclosure of Invention
The invention provides a temperature anomaly defect detection and positioning method and a system for overcoming the defects of high temperature anomaly defect judgment and insufficient positioning efficiency caused by high registration difficulty of temperature anomaly defect detection and positioning feature points and high camera position requirement in the prior art.
The method comprises the following steps:
s1: fixing the relative positions of the infrared thermal imaging temperature measuring module and the visible light imaging module, wherein the mirror surface of the infrared thermal imaging temperature measuring module is relatively parallel to the mirror surface of the visible light imaging module; the infrared thermal imaging temperature measuring module collects infrared thermal image data, and the visible light imaging module collects visible light image data; transmitting the infrared heat map data and the visible light image data to an image analysis positioning module;
s2: the image analysis positioning module receives infrared heat map data and visible light image data, and obtains temperature abnormality detection and positioning results through image analysis processing;
s3: and transmitting the image and text information of the temperature abnormality detection positioning result to a result display module for display, and transmitting the temperature abnormality detection signal to an alarm module for alarm.
Preferably, the relative parallelism of S1, allows a certain deflection angle, and in order to prevent the collected visible light image and infrared thermal map from having an excessive rotation deflection angle, the deflection angle range is preferably: 0-10 deg.
Preferably, S2 comprises the steps of:
s21: using an infrared heat map to judge temperature abnormality according to an input temperature threshold; if the temperature abnormality region does not exist, judging that no temperature abnormality defect occurs, outputting a shooting original picture, and sending a temperature abnormality defect-free signal; if the temperature abnormal region exists, judging that the temperature abnormal defect exists, sending a temperature abnormal defect alarm signal, and storing the position of the temperature abnormal region for the next image processing call;
s22: performing image preprocessing on the visible light image and the infrared heat map;
s23: performing wavelet multi-scale decomposition on the preprocessed visible light image by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component image obtained after the wavelet multi-scale decomposition to obtain each layer of gradient image of the visible light image;
performing wavelet multi-scale decomposition on the preprocessed infrared heat map by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component image obtained after the wavelet multi-scale decomposition to obtain each layer of gradient image of the infrared heat map;
s24: and (3) positioning a temperature abnormal region through a template matching mode on the gradient image of the multilayer visible light and the gradient image of the multilayer infrared heat map.
Preferably, the temperature anomaly in S21 specifically includes: the temperature is too high and the temperature is too low.
Preferably, preprocessing the visible light image in S22 specifically includes: graying and denoising the image; converting the obtained visible light image into a gray scale image; in order to remove noise while maintaining the image edge detail information as much as possible, an adaptive median filter is used for denoising.
Preferably, the preprocessing the infrared heat map in S22 specifically includes: morphological filtering and denoising; performing background suppression on the obtained infrared heat map by using morphological filtering; the morphological filtering comprises selecting structural elements, eliminating bright noise by using open operation, eliminating dark noise by using closed operation, and obtaining an image after suppressing a background by using difference between an original image and an image obtained after processing; in order to remove noise while maintaining the image edge detail information as much as possible, an adaptive median filter is used for denoising.
Preferably, S24 comprises the steps of:
s241: using the alignment degree as a similarity criterion, using a template matching mode and a sequence of coarse and fine steps and small steps and big steps as a matching strategy, traversing the gradient image obtained by processing, and guiding a fine resolution matching search process by using a coarse resolution result to obtain an optimal geometric transformation relation between two images to be registered;
s242: and according to the geometric transformation relation obtained in the step S241, calculating the position relation and the range corresponding to the infrared heat map and the visible light image, determining the position and the region of the temperature abnormal region obtained in the step S21 in the visible light image through affine transformation, and realizing the positioning of the temperature abnormal region in the visible light image.
Preferably, the specific calculation method of the alignment degree is as follows:
image I A (x, y) and image I B The magnitude of (x, y) is m×n, and H is defined for each gray level n=k (k=0 to 255) A (n) and H B (n) each represents an image I A (x, y) and image I B The number of pixels in (x, y) having a gray value k, so that the occurrence probabilities of the gray value k in the two images are respectively:
for image I A Each gray level n of (x, y) now defines an image I B (x, y) relative to image I A The gray mean and variance of the corresponding pixel set with (x, y) gray value n are:
probability P of occurrence with gray value n A (n) vs. sigma 2 A,B (n) weighted averaging to obtain the image-based I A The expected variance of (x, y):
can be obtained by the same way
And->
Defining the interaction variance of the two images as follows:
middle sigma 2 A Sum sigma 2 B Respectively are images I A (x, y) and image I B (x, y) variance;
therefore, the alignment degree is defined as:
preferably, when the alignment degree is calculated, the quick traversing method of the alignment degree is as follows:
first, a zero-digit group M of 1X 256 dimensions is established A 、M B The method comprises the steps of carrying out a first treatment on the surface of the Then traversing the image to obtain an image I B (x, y) a gray value k at (x, y); then M corresponding to the gray level k A (k) Performing image I A The gray values of (x, y) at (x, y) are accumulated to obtain an image I B (x, y) relative to image I A Corresponding set of pixels M with (x, y) gray values k A (k) The image I can be obtained after one traversal is finished B (x, y) relative to image I A N gray scale corresponding pixel sets M of (x, y) gray scale values A The method comprises the steps of carrying out a first treatment on the surface of the Image I can be obtained in the same way A (x, y) relative to image I B N gray scale corresponding pixel sets M of (x, y) gray scale values B 。
Preferably, calculating the alignment degree of gradient images of the two images to be registered from 0 layer to n layers under different translation, rotation and scaling conditions; the visible light n-layer gradient image is used as a reference image to be unchanged, the infrared heat image n-layer gradient image is used as an image to be registered, and the image to be registered is subjected to different translation, rotation and scaling transformation and then is subjected to calculation alignment degree calculation with the reference image; searching four parameters (X translation X) of rigid affine transformation that maximize alignment using a search algorithm n Translation of Y n Angle of rotation theta n Scaling factor ρ n ) The method comprises the steps of carrying out a first treatment on the surface of the And then, n layers of output parameters are used as n-1 layers of input parameters, a search algorithm is used for searching in a certain range near the optimal matching position, searching in the whole image range is avoided, useless search intervals are greatly reduced, the operation amount is effectively reduced, and the registration efficiency is improved. Searching for four parameters (X translation X) of the rigid affine transformation that maximize alignment n-1 Translation of Y n-1 Angle of rotation theta n-1 Scaling factor ρ n-1 n) until a registration result of 0 layers (original pictures) is output, and the registration result is used as the optimal geometric transformation relation between two images to be registered.
The system of the invention can realize the temperature anomaly defect detection and positioning method, which comprises the following steps: the system comprises an infrared thermal imaging temperature measuring module, a visible light imaging module and an information interaction system;
the information interaction system comprises a parameter setting module, a result display module, an alarm module and an image analysis positioning module;
the infrared thermal imaging temperature measurement module collects an on-site infrared thermal map and transmits infrared thermal map data to the image analysis positioning module;
the visible light imaging module is used for acquiring a field visible light image and transmitting visible light image data to the image analysis positioning module;
the parameter setting module is used for setting parameters of the infrared thermal imaging temperature measuring module, parameters of the visible light imaging module, results display and alarm information by a user;
the result display module is used for displaying a temperature abnormality detection positioning result image and prompting related information;
the alarm module sends an alarm prompt to the outside according to the temperature abnormality detection positioning result;
the image analysis positioning module receives visible light image data and infrared heat map data, and transmits image text information and alarm signals of the obtained temperature abnormality detection positioning result to the result display module and the alarm module respectively through image analysis processing.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that: the temperature anomaly defect judging and positioning efficiency is higher. The registration of the feature points is complicated, and the realization efficiency of the registration is low. The registration by using the camera calibration mode has high requirements on the position and the installation of the camera. The gradient information is used for matching the template in combination with the alignment degree, so that complicated operation and processing procedures of extracting the characteristic points of the infrared heat map and the visible light image and registering the characteristic points are avoided, and high requirements of camera position and installation are avoided; and the multi-scale template matching is used, so that useless search intervals are greatly reduced, the operand is effectively reduced, and the registration efficiency is further improved. The method comprises the following steps:
(1) The invention uses multi-scale gradient information to combine with alignment degree to match the template, thereby avoiding complex operation and processing procedures of feature point extraction and feature registration and high requirements of camera position and installation;
(2) The invention uses a quick traversing method when calculating the alignment degree, can complete the calculation of the pixel sets corresponding to all gray levels by one-time traversing, and avoids the problems of large calculated amount and low efficiency caused by the need of multiple times of traversing of a plurality of gray levels.
(3) According to the invention, a multi-scale template matching is used, the sequence of firstly thickening, secondly refining and firstly shrinking and secondly enlarging is used as a matching strategy, and the result of the coarse resolution is used for guiding the matching searching process of the fine resolution, so that useless searching intervals are greatly reduced, and the operand is effectively reduced;
(4) The invention can customize the input temperature threshold, and can customize and identify the temperature abnormal region in the visible light image according to the input temperature threshold, thereby facilitating the check and analysis.
Drawings
Fig. 1 is a flowchart of a temperature anomaly detection and localization method according to embodiment 1.
Fig. 2 is an image processing flow chart of the image processing module.
Fig. 3 is a schematic diagram of a temperature anomaly detection and location system according to embodiment 2.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the present patent;
for the purpose of better illustrating the embodiments, certain elements of the drawings may be omitted, enlarged or reduced and do not represent the actual product dimensions;
it will be appreciated by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical scheme of the invention is further described below with reference to the accompanying drawings and examples.
Example 1:
the embodiment provides a temperature anomaly defect detection and positioning method, as shown in fig. 1, comprising the following steps:
s1: fixing the relative positions of the infrared thermal imaging temperature measuring module and the visible light imaging module, wherein the mirror surfaces of the infrared thermal imaging temperature measuring module and the visible light imaging module are relatively parallel; the infrared thermal imaging temperature measuring module collects infrared thermal image data, and the visible light imaging module collects visible light image data. Transmitting the infrared heat map data and the visible light image data to an image analysis positioning module;
in this embodiment, the infrared thermal imaging temperature measuring module is a thermal infrared imager, and the visible light imaging module is a visible light camera.
S2: the image analysis positioning module receives infrared heat map data and visible light image data, and obtains temperature abnormality detection and positioning results through image analysis processing;
s3: and transmitting the image and text information of the temperature abnormality detection positioning result to a result display module for display, and transmitting the temperature abnormality detection signal to an alarm module for alarm.
Further, the step S1 includes that, to ensure that the infrared field of view coincides with the visible light field of view as much as possible, the infrared image field of view should be included in the visible light image field of view.
Further, the relative parallelism of S1, a certain deflection angle is allowed, and in order to prevent the collected visible light image and infrared thermal map from having an excessive rotation deflection angle, the deflection angle range is preferably: 0-10 deg.
As shown in fig. 2, the image processing flow S2 of the image processing module specifically includes:
s21: and carrying out temperature abnormality judgment according to the input temperature threshold by using an infrared heat map. If the temperature abnormality region does not exist, judging that no temperature abnormality defect occurs, outputting a shooting original picture, and sending a temperature abnormality defect-free signal; if the temperature abnormal region exists, judging that the temperature abnormal defect exists, sending a temperature abnormal defect alarm signal, and storing the position of the temperature abnormal region for the next image processing call;
further, the temperature anomaly in S21 specifically includes: the temperature is too high and the temperature is too low.
S22: and carrying out image preprocessing on the visible light image and the infrared heat map.
Further, the preprocessing the visible light image in S22 specifically includes: graying and denoising the image, and converting the obtained visible light image into a gray image; in order to remove noise while maintaining the image edge detail information as much as possible, an adaptive median filter is used for denoising.
Further, the preprocessing the ir heat map in S22 specifically includes: morphological filtering and denoising; performing background suppression on the obtained infrared heat map by using morphological filtering; the morphological filtering comprises selecting proper structural elements, eliminating bright noise by using open operation, eliminating dark noise by using closed operation, and obtaining an image after background inhibition by using difference between an original image and an image obtained after processing; in order to remove noise while maintaining the image edge detail information as much as possible, an adaptive median filter is used for denoising.
S23: in order to reduce the calculated amount and improve the calculation efficiency as much as possible, wavelet transformation is used for carrying out wavelet multi-scale decomposition on the preprocessed visible light image, and then gradient extraction is carried out on each layer of low-frequency component image obtained after the wavelet multi-scale decomposition, so as to obtain each layer of gradient image of the visible light image. And carrying out wavelet multi-scale decomposition on the preprocessed infrared heat map by using wavelet transformation, and carrying out gradient extraction on each layer of low-frequency component image obtained after the wavelet multi-scale decomposition to obtain each layer of gradient image of the infrared heat map.
Furthermore, the wavelet transformation has good time-frequency localization characteristics and multi-scale analysis capability, and original image detail information can be kept as much as possible when the scale is changed.
Further, the wavelet multi-scale decomposition is preferably 2 to 3 layers. In this embodiment, 2 layers are selected. The infrared heat map and the visible light image are decomposed into a two-layer wavelet pyramid with decreasing precision and decreasing size through wavelet transformation, and bin 3.7 is selected as a wavelet base of the wavelet transformation.
It should be noted that, the parameters such as the number of decomposition layers and the wavelet base for performing wavelet multi-scale decomposition on the infrared image and the visible light image provided in this embodiment are not unique, and are not particularly limited herein, and those skilled in the art may select and use according to actual situations.
S24: and (3) carrying out temperature anomaly region positioning on the gradient image of the multilayer visible light and the gradient image of the multilayer infrared heat map through template matching.
S241: and traversing the gradient image obtained by processing by using the alignment degree as a similarity criterion and using a template matching mode and a sequence of coarse and fine steps and small steps and large steps as a matching strategy, and guiding a fine resolution matching search process by using a coarse resolution result to obtain an optimal geometric transformation relation between two images to be registered.
Further, the alignment means that the gray value of each gray value of one image is most stable in the pixel position of the other image corresponding to each gray value, and is expressed as the smallest variance mathematically.
Further, the specific calculation method for calculating the alignment degree comprises the following steps:
image I A (x, y) and image I B The magnitude of (x, y) is m×n, and H is defined for each gray level n=k (k=0 to 255) A (n) and H B (n) each represents an image I A (x, y) and image I B The number of pixels in (x, y) having a gray value k, so that the occurrence probabilities of the gray value k in the two images are respectively:
for image I A Each gray level n of (x, y) now defines an image I B (x, y) relative to image I A The gray mean and variance of the corresponding pixel set with (x, y) gray value n are:
probability P of occurrence with gray value n A (n) vs. sigma 2 A,B (n) weighted averaging to obtain the image-based I A The expected variance of (x, y):
can be obtained by the same way
And->
Defining the interaction variance of the two images as follows:
middle sigma 2 A Sum sigma 2 B Respectively are images I A (x, y) and image I B Variance of (x, y).
The interaction variance reflects the mutual correspondence stability of the gradient images of the two images, and it can be seen that the more similar the contents of the two images are, the smaller the interaction variance of the two images is. For convenience of description, the alignment degree is defined as follows:
furthermore, in the rapid traversal method, when the alignment degree is calculated, the most complicated and time-consuming calculation is as follows: obtaining image I B (x, y) relative to image I A And a set of pixels corresponding to n gray levels of the (x, y) gray values. The general operation needs to perform n gray levels and n times of traversal acquisition. A method for solving all the pixel sets corresponding to the gray level can be completed through one traversal is provided. First, a zero-number group M of 1X 256 dimensions (corresponding to gray scales 0-255) is established A 、M B The method comprises the steps of carrying out a first treatment on the surface of the Then traversing the image to obtain an image I B (x, y) is represented by the formula (x,y) the gray value k at y); then M corresponding to the gray level k A (k) Performing image I A The gray values of (x, y) at (x, y) are accumulated to obtain an image I B (x, y) relative to image I A Corresponding set of pixels M with (x, y) gray values k A (k) The image I can be obtained after one traversal is finished B (x, y) relative to image I A N gray scale corresponding pixel sets M of (x, y) gray scale values A The method comprises the steps of carrying out a first treatment on the surface of the Image I can be obtained in the same way A (x, y) relative to image I B N gray scale corresponding pixel sets M of (x, y) gray scale values B 。
Further, calculating the alignment degree of gradient images of the two images to be registered of the 0 layer (original image) to the 2 layer (original image) under different translation, rotation and scaling conditions; and (3) taking the visible light 2-layer gradient image as a reference image to be unchanged, taking the infrared heat figure 2-layer gradient image as an image to be registered, carrying out different translation, rotation and scaling transformation on the image to be registered, comparing the image to be registered with the reference image, and calculating the alignment degree. A Powell search algorithm is selected to search four parameters (X translation X) of rigid affine transformation which enables the alignment degree to reach the maximum 2 Translation of Y 2 Angle of rotation theta 2 Scaling factor ρ 2 ). And then searching in a certain range near the optimal matching position by using a Powell searching algorithm according to the 2-layer output parameter as a 1-layer input parameter, avoiding searching in the whole graph range, greatly reducing useless searching intervals, effectively reducing the operand and improving the registration efficiency. Searching for four parameters (X translation X) of the rigid affine transformation that maximize alignment 1 Translation of Y 1 Angle of rotation theta 1 Scaling factor ρ 1 ) And outputting a registration result of 0 layers until the registration result is output and taking the registration result as the optimal geometric transformation relation between the two images to be registered.
It should be noted that, the selection of the template matching search algorithm provided in this embodiment is not unique, and is not limited herein, and those skilled in the art may select and use the algorithm according to actual situations.
S242: according to the geometric transformation relation obtained in S241, the position relation and the range corresponding to the infrared heat map and the visible light image can be calculated, the position and the region of the temperature abnormal region obtained in S21 in the visible light image are determined through affine transformation, and the positioning of the temperature abnormal region in the visible light image is realized.
Example 2:
the embodiment provides a temperature anomaly defect detection and positioning system, as shown in fig. 3, including: the system comprises an infrared thermal imaging temperature measuring module, a visible light imaging module and an information interaction system, wherein the information interaction system comprises a parameter setting module, a result display module, an alarm module and an image analysis positioning module; the infrared thermal imaging temperature measurement module collects an on-site infrared thermal map and transmits infrared thermal map data to the image analysis positioning module; the visible light imaging module is used for acquiring a field visible light image and transmitting visible light image data to the image analysis positioning module; the parameter setting module is used for setting parameters of the infrared thermal imaging temperature measuring module, parameters of the visible light imaging module, results display and alarm information by a user; the result display module is used for displaying a temperature abnormality detection positioning result image and prompting related information; the alarm module sends an alarm prompt to the outside according to the temperature abnormality detection positioning result; the image analysis positioning module receives visible light image data and infrared heat map data, and transmits image text information and alarm signals of the obtained temperature abnormality detection positioning result to the result display module and the alarm module respectively through image analysis processing.
In this embodiment, the infrared thermal imaging temperature measuring module is a thermal infrared imager, and the visible light imaging module is a visible light camera.
According to the number of the detection points and the complexity of the environment, the inspection robot, the fixed cradle head and the handheld binocular vision equipment can be used for image acquisition as required.
The terms describing the positional relationship in the drawings are merely illustrative, and are not to be construed as limiting the present patent;
it is to be understood that the above examples of the present invention are provided by way of illustration only and not by way of limitation of the embodiments of the present invention. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the invention are desired to be protected by the following claims.
Claims (6)
1. A method for detecting and locating a temperature anomaly defect, the method comprising the steps of:
s1: fixing the relative positions of the infrared thermal imaging temperature measuring module and the visible light imaging module, wherein the mirror surfaces of the infrared thermal imaging temperature measuring module and the visible light imaging module are relatively parallel; the infrared thermal imaging temperature measuring module collects infrared thermal image data, and the visible light imaging module collects visible light image data; transmitting the infrared heat map data and the visible light image data to an image analysis positioning module;
s2: the image analysis positioning module receives infrared heat map data and visible light image data, and obtains temperature abnormality detection and positioning results through image analysis processing;
s2 comprises the following steps:
s21: using an infrared heat map to judge temperature abnormality according to an input set temperature threshold; if the temperature abnormality region does not exist, judging that no temperature abnormality defect occurs, outputting a shooting original picture, and sending a temperature abnormality defect-free signal; if the temperature abnormal region exists, judging that the temperature abnormal defect exists, sending a temperature abnormal defect alarm signal, and storing the position of the temperature abnormal region for the next image processing call;
s22: performing image preprocessing on the visible light image and the infrared heat map;
s23: performing wavelet multi-scale decomposition on the preprocessed visible light image by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component image obtained after the wavelet multi-scale decomposition to obtain each layer of gradient image of the visible light image;
performing wavelet multi-scale decomposition on the preprocessed infrared heat map by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component image obtained after the wavelet multi-scale decomposition to obtain each layer of gradient image of the infrared heat map;
s24: carrying out temperature anomaly region positioning on the gradient image of the multilayer visible light and the gradient image of the multilayer infrared heat map in a template matching mode;
s24 comprises the following steps:
s241: using the alignment degree as a similarity criterion, using a template matching mode and a sequence of coarse and fine steps and small steps and big steps as a matching strategy, traversing the gradient image obtained by processing, and guiding a fine resolution matching search process by using a coarse resolution result to obtain an optimal geometric transformation relation between two images to be registered;
s242: according to the geometric transformation relation obtained in the step S241, the position relation and the range corresponding to the infrared heat map and the visible light image are calculated, the position and the region of the temperature abnormal region obtained in the step S21 in the visible light image are determined through affine transformation, and the positioning of the temperature abnormal region in the visible light image is realized;
the specific calculation method of the alignment degree comprises the following steps:
image I A (x, y) and image I B (x, y) is m×n, and H is defined for each gray level n=k, k=0 to 255 A (n) and H B (n) each represents an image I A (x, y) and image I B The number of pixels in (x, y) having a gray value k, so that the occurrence probabilities of the gray value k in the two images are respectively:
for image I A Each gray level n of (x, y) now defines an image I B (x, y) relative to image I A The gray mean and variance of the corresponding pixel set with (x, y) gray value n are:
probability P of occurrence with gray value n A (n) vs. sigma 2 A,B (n) weighted averaging to obtain the image-based I A The expected variance of (x, y):
can be obtained by the same way
And->
Defining the interaction variance of the two images as follows:
middle sigma 2 A Sum sigma 2 B Respectively are images I A (x, y) and image I B (x, y) variance;
therefore, the alignment degree is defined as:
when the alignment degree is calculated, the quick traversing method of the alignment degree comprises the following steps:
first, a zero-digit group M of 1X 256 dimensions is established A 、M B The method comprises the steps of carrying out a first treatment on the surface of the Then traversing the image to obtain an image I B (x, y) a gray value k at (x, y); then M corresponding to the gray level k A (k) Performing image I A The gray values of (x, y) at (x, y) are accumulated to obtain an image I B (x, y) relative to image I A Corresponding set of pixels M with (x, y) gray values k A (k) The image I can be obtained after one traversal is finished B (x, y) relative to image I A N gray scale corresponding pixel sets M of (x, y) gray scale values A The method comprises the steps of carrying out a first treatment on the surface of the Image I can be obtained in the same way A (x, y) relative to image I B N gray scale corresponding pixel sets M of (x, y) gray scale values B 。
2. The temperature anomaly defect detection localization method of claim 1, further comprising S3: and transmitting the image and text information of the temperature abnormality detection and positioning result to a result display module for display, and transmitting the temperature abnormality detection signal to an alarm module for alarm.
3. The method for detecting and locating a temperature anomaly defect according to claim 1, wherein preprocessing the visible light image in S22 specifically comprises: graying and denoising the image; converting the obtained visible light image into a gray scale image; in order to remove noise while maintaining the image edge detail information as much as possible, an adaptive median filter is used for denoising.
4. The method for detecting and locating a temperature anomaly defect according to claim 3, wherein the preprocessing of the infrared heat map in S22 specifically comprises: morphological filtering and denoising; performing background suppression on the obtained infrared heat map by using morphological filtering; the morphological filtering comprises selecting structural elements, eliminating bright noise by using open operation, eliminating dark noise by using closed operation, and obtaining an image after suppressing a background by using difference between an original image and an image obtained after processing; in order to remove noise while maintaining the image edge detail information as much as possible, an adaptive median filter is used for denoising.
5. The method for detecting and locating a temperature anomaly defect according to claim 1, which comprisesThe method is characterized by calculating the alignment degree of gradient images of 0-n layers of images to be registered under different translation, rotation and scaling conditions; the visible light n-layer gradient image is used as a reference image to be unchanged, the infrared heat image n-layer gradient image is used as an image to be registered, and alignment degree calculation is carried out on the image to be registered after different translation, rotation and scaling transformation are carried out on the image to be registered; searching four parameters of rigid affine transformation for maximum alignment using a search algorithm, i.e. X translation X n Translation of Y n Angle of rotation theta n Scaling factor ρ n The method comprises the steps of carrying out a first treatment on the surface of the Then, n layers of output parameters are used as n-1 layers of input parameters, searching is carried out in a certain range near the optimal matching position by using a searching algorithm, searching in the whole image range is avoided, useless searching intervals are greatly reduced, the operand is effectively reduced, and the registration efficiency is improved; searching four parameters of rigid affine transformation for maximum alignment, i.e. X translation X n-1 Translation of Y n-1 Angle of rotation theta n-1 Scaling factor ρ n-1 And n, outputting a registration result of 0 layers until the registration result is output, wherein the registration result is used as the optimal geometric transformation relation between two images to be registered.
6. A system applying the temperature anomaly defect detection localization method of any one of claims 1-5, the system comprising:
the system comprises an infrared thermal imaging temperature measuring module, a visible light imaging module and an information interaction system;
the information interaction system comprises a parameter setting module, a result display module, an alarm module and an image analysis positioning module;
the infrared thermal imaging temperature measurement module collects an on-site infrared thermal map and transmits infrared thermal map data to the image analysis positioning module;
the visible light imaging module is used for acquiring a field visible light image and transmitting visible light image data to the image analysis positioning module;
the parameter setting module is used for setting parameters of the infrared thermal imaging temperature measuring module, parameters of the visible light imaging module, results display and alarm information by a user;
the result display module is used for displaying a temperature abnormality detection positioning result image and prompting related information;
the alarm module sends an alarm prompt to the outside according to the temperature abnormality detection positioning result;
the image analysis positioning module receives visible light image data and infrared heat map data, and transmits image text information and alarm signals of the obtained temperature abnormality detection positioning result to the result display module and the alarm module respectively through image analysis processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911244645.9A CN110942458B (en) | 2019-12-06 | 2019-12-06 | Temperature anomaly defect detection and positioning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911244645.9A CN110942458B (en) | 2019-12-06 | 2019-12-06 | Temperature anomaly defect detection and positioning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110942458A CN110942458A (en) | 2020-03-31 |
| CN110942458B true CN110942458B (en) | 2023-05-16 |
Family
ID=69910082
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911244645.9A Active CN110942458B (en) | 2019-12-06 | 2019-12-06 | Temperature anomaly defect detection and positioning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110942458B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111242144B (en) * | 2020-04-26 | 2020-08-21 | 北京邮电大学 | Method and device for detecting abnormality of power grid equipment |
| CN111553910B (en) * | 2020-05-06 | 2022-03-25 | 哈尔滨工业大学 | Infrared image-based thermal fault detection method for electrical equipment of large experimental device |
| CN111795745A (en) * | 2020-05-29 | 2020-10-20 | 红鼎互联(广州)信息科技有限公司 | Temperature calibration processing method for infrared image |
| CN112540635B (en) * | 2020-12-03 | 2021-08-17 | 华润三九（枣庄）药业有限公司 | Traditional chinese medicine production intelligence quality control system based on artificial intelligence |
| CN112539843B (en) * | 2020-12-11 | 2023-06-02 | 平安科技（深圳）有限公司 | Method and device for detecting temperature and computer equipment |
| CN113674271B (en) * | 2021-09-06 | 2022-06-07 | 广东康德威电气股份有限公司 | Transformer monitoring system based on cloud computing |
| CN114577405A (en) * | 2022-02-28 | 2022-06-03 | 辽宁石油化工大学 | Urban underground sewage pipeline defect detection device based on infrared imaging technology |
| CN115375588B (en) * | 2022-10-25 | 2023-02-07 | 山东旗胜电气股份有限公司 | Power grid transformer fault identification method based on infrared imaging |
| CN115661016B (en) * | 2022-12-08 | 2023-04-18 | 瑞纳智能设备股份有限公司 | Fault monitoring method and system of heat exchange station and embedded image diagnosis control platform |
| CN116739994A (en) * | 2023-05-18 | 2023-09-12 | 山东新宝龙工业科技有限公司 | Digital conveyer belt graphic data monitoring system with early warning function |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025648A (en) * | 2017-03-20 | 2017-08-08 | 中国人民解放军空军工程大学 | A kind of board failure infrared image automatic testing method |
| CN107253485A (en) * | 2017-05-16 | 2017-10-17 | 北京交通大学 | Foreign matter invades detection method and foreign matter intrusion detection means |
| CN109410159A (en) * | 2018-09-11 | 2019-03-01 | 上海创客科技有限公司 | Binocular visible light and infrared thermal imaging complex imaging system, method and medium |
-
2019
- 2019-12-06 CN CN201911244645.9A patent/CN110942458B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025648A (en) * | 2017-03-20 | 2017-08-08 | 中国人民解放军空军工程大学 | A kind of board failure infrared image automatic testing method |
| CN107253485A (en) * | 2017-05-16 | 2017-10-17 | 北京交通大学 | Foreign matter invades detection method and foreign matter intrusion detection means |
| CN109410159A (en) * | 2018-09-11 | 2019-03-01 | 上海创客科技有限公司 | Binocular visible light and infrared thermal imaging complex imaging system, method and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110942458A (en) | 2020-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110942458B (en) | Temperature anomaly defect detection and positioning method and system | |
| EP2417419B1 (en) | Digital optical comparator and corresponding digital comparison method | |
| CN109919933B (en) | VR device, picture detection method and device thereof, and computer-readable storage medium | |
| JP2003254748A (en) | Stereo image characteristic inspection system | |
| CN101996398A (en) | Image matching method and equipment for wafer alignment | |
| US8879868B2 (en) | Image processing method for converting an annual image | |
| JP5172643B2 (en) | Printed matter contamination degree determination apparatus and printed matter contamination degree determination method | |
| CN110322485B (en) | Rapid image registration method of heterogeneous multi-camera imaging system | |
| KR20140075042A (en) | Apparatus for inspecting of display panel and method thereof | |
| CN111145198B (en) | Non-cooperative target motion estimation method based on rapid corner detection | |
| CN114897705A (en) | Unmanned aerial vehicle remote sensing image splicing method based on feature optimization | |
| CN104966283A (en) | Imaging layered registering method | |
| CN106709941A (en) | Key point screening method for spectrum image sequence registration | |
| JP2010217984A (en) | Image detector and image detection method | |
| JP2005345290A (en) | Streak-like flaw detecting method and streak-like flaw detector | |
| KR20140038749A (en) | Apparatus and method for correcting image distortion | |
| CN104820972B (en) | A kind of infrared image ME noise remove methods based on in-orbit statistic of classification | |
| CN109799068B (en) | Unstable blind pixel detection method and system | |
| TW201445458A (en) | Testing device and method for camera | |
| CN113008470B (en) | Gas leakage detection device and gas leakage detection method | |
| CN113516654A (en) | Method and system for identifying abnormal part of inner wall of core hole based on vision | |
| CN104112271A (en) | Detection method and system for housing side defect of mobile terminal | |
| CN111062887B (en) | Image definition judging method based on improved Retinex algorithm | |
| CN117612018B (en) | Intelligent discrimination method for optical remote sensing load astigmatism | |
| JP4893938B2 (en) | Defect inspection equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231025 Address after: Building B, 3-1B, Phase II, Diejin Industrial Zone, Jinping District, Shantou, Guangdong, 515000 Patentee after: SHANTOU A-PLUX OPTOELECTRONIC TECHNOLOGY CO.,LTD. Address before: 515063 No. 243, University Road, Shantou, Guangdong Patentee before: SHANTOU University |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240124 Address after: Building 3, 6th Floor, Financial Center, Dongshan Street Office, Nankang District, Ganzhou City, Jiangxi Province, 341000 Patentee after: Jiangxi Xinkang Technology Co.,Ltd. Country or region after: China Address before: Building B, 3-1B, Phase II, Diejin Industrial Zone, Jinping District, Shantou, Guangdong, 515000 Patentee before: SHANTOU A-PLUX OPTOELECTRONIC TECHNOLOGY CO.,LTD. Country or region before: China |