CN110942458A - Temperature anomaly defect detection and positioning method and system - Google Patents
Temperature anomaly defect detection and positioning method and system Download PDFInfo
- Publication number
- CN110942458A CN110942458A CN201911244645.9A CN201911244645A CN110942458A CN 110942458 A CN110942458 A CN 110942458A CN 201911244645 A CN201911244645 A CN 201911244645A CN 110942458 A CN110942458 A CN 110942458A
- Authority
- CN
- China
- Prior art keywords
- image
- visible light
- module
- temperature
- positioning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 43
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000001514 detection method Methods 0.000 title claims abstract description 34
- 238000010191 image analysis Methods 0.000 claims abstract description 28
- 238000003384 imaging method Methods 0.000 claims abstract description 27
- 238000009529 body temperature measurement Methods 0.000 claims abstract description 24
- 238000001931 thermography Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 22
- 230000002159 abnormal effect Effects 0.000 claims description 37
- 230000009466 transformation Effects 0.000 claims description 32
- 238000013519 translation Methods 0.000 claims description 18
- 238000000354 decomposition reaction Methods 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 15
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 12
- 230000003993 interaction Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 9
- 230000000877 morphologic effect Effects 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 230000005856 abnormality Effects 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000010845 search algorithm Methods 0.000 claims description 6
- 230000001629 suppression Effects 0.000 claims description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20036—Morphological image processing
Abstract
The invention relates to a method and a system for detecting and positioning temperature anomaly defects, wherein the system comprises the following components: the system comprises an infrared thermal imaging temperature measurement module, a visible light imaging module, a parameter setting module, a result display module, an alarm module and an image analysis positioning module; the method comprises the following steps: s1: fixing the relative positions of the infrared thermal imaging temperature measurement module and the visible light imaging module, wherein the mirror surfaces of the infrared thermal imaging temperature measurement module and the visible light imaging module are relatively parallel; the infrared thermal imaging temperature measurement module collects infrared thermal image data, and the visible light imaging module collects visible light image data; the data of the two are transmitted to an image analysis positioning module; s2: the image analysis positioning module receives infrared heat image data and visible light image data, and obtains temperature anomaly detection and positioning results through image analysis and processing; s3: and transmitting the image and the character information of the temperature abnormity detection positioning result to a result display module for displaying, and transmitting a temperature abnormity detection signal to an alarm module for alarming. The invention can improve the efficiency of judging and positioning the temperature abnormity defects.
Description
Technical Field
The invention relates to the field of temperature anomaly defect detection and positioning, in particular to a temperature anomaly defect detection and positioning method and system.
Background
And detecting and positioning the temperature abnormal defect by using an image, generally acquiring temperature data by using a thermal infrared imager to obtain an infrared temperature chart, and positioning the temperature abnormal defect. However, most thermal infrared imagers have low resolution which is far lower than that of a common visible light camera, which causes serious loss of details of the thermal infrared image and image blurring. At present, the temperature abnormal defect is generally positioned according to the shooting position of an inspection robot or the shooting position of a technician to roughly determine the temperature abnormal area, and the specific position of the temperature abnormality can be determined only by the remote or further inspection in the field of workers familiar with the field and infrared heat maps. Therefore, the method has the advantages of large workload and low efficiency, and simultaneously requires the working personnel to have better professional skills and abundant practical experience.
Chinese patent CN 110223330 a proposes a registration method and system for visible light and infrared images. The patent collects visible light images and infrared images of the power equipment; carrying out primary registration on the infrared image and the visible light image; obtaining an affine transformation matrix set by randomly selecting three pairs of point pairs from the primary registration point pairs, obtaining a set which enables the image edge correlation degree to be highest in the affine transformation matrix set to obtain the optimal three pairs of registration point pairs, and constructing a registration point neighborhood; and traversing and searching the sub-pixel points with the highest affine edge correlation degree in the neighborhood of the registration point by using the selected sub-pixel units, and finally obtaining the registration point pair. The patent uses corner detection to extract features and introduces edge correlation to calculate registration point pairs. The feature point pairing is complicated, and the image registration is easy to have large deviation due to local obvious features.
Chinese patent CN 110335271 a proposes an infrared detection method and device for electrical component failure. The method comprises the steps of collecting infrared and visible light images of a normal electrical component as standard images, and collecting the infrared and visible light images of an electrical component to be detected during detection; correspondingly processing the infrared images and the visible light images of the normal component and the electrical component to be detected so as to realize the complete matching of the visible light images and the infrared images; and carrying out difference comparison analysis on gray level images obtained by carrying out graying processing on the infrared images of the two completely matched electrical components to be detected and normal to determine the abnormal condition of the electrical component to be detected. The patent realizes that the complete matching of the visible light image and the infrared image can be realized only by calibration, and when the relative position of the infrared thermal imager and the visible light camera deviates due to the deviation caused in daily use, the complete matching can be normally completed only by calibrating or adjusting the relative position of the camera again.
At present, the following defects mainly exist in the detection and positioning of temperature anomaly defects:
(1) the registration difficulty of the feature points is large. Due to the difference of imaging modes and imaging platforms among sensors, an infrared image and a visible light image obtained by imaging the same area at the same time have larger difference. The infrared image and the visible light image have great difference in resolution and color, so that the registration of the infrared image and the visible light image is difficult to realize; in terms of resolution, the infrared image is mostly far lower than that of a common visible light image, which causes serious loss of gray level details of the infrared image, image blurring, and a large difference from clear textures in the visible light gray level image. The feature point extraction can be realized quickly, but the high-precision feature point pairing is realized, so that the complex descriptor establishment is often involved, and the influence caused by the transformation of size, resolution, direction and the like is solved.
(2) The camera position requirement is high. The registration of the visible light image and the infrared image is realized quickly and simply by adopting a camera calibration mode. However, when the relative position between the infrared thermal imager and the visible light camera is shifted due to normal vibration in daily use, the position mapping will be deviated, and the image registration can be normally completed only by recalibrating or adjusting the relative position of the camera.
Disclosure of Invention
The invention provides a method and a system for detecting and positioning temperature anomaly defects, aiming at overcoming the defects of high registration difficulty of the temperature anomaly defect detection and positioning characteristic points and low temperature anomaly defect judgment and positioning efficiency caused by high camera position requirements in the prior art.
The method comprises the following steps:
s1: fixing the relative positions of the infrared thermal imaging temperature measurement module and the visible light imaging module, wherein the mirror surface of the infrared thermal imaging temperature measurement module is relatively parallel to the mirror surface of the visible light imaging module; the infrared thermal imaging temperature measurement module collects infrared thermal image data, and the visible light imaging module collects visible light image data; transmitting the infrared heat map data and the visible light image data to an image analysis positioning module;
s2: the image analysis positioning module receives the infrared heat image data and the visible light image data, and obtains temperature anomaly detection and positioning results through image analysis processing;
s3: and transmitting the image and the character information of the temperature abnormity detection positioning result to a result display module for displaying, and transmitting a temperature abnormity detection signal to an alarm module for alarming.
Preferably, the relative parallelism of S1 allows for a certain declination angle, and to prevent excessive rotational declination of the captured visible light image and infrared thermal image, the declination angle range is preferably: 0 to 10 degrees.
Preferably, S2 includes the steps of:
s21: judging temperature abnormality according to an input temperature threshold value by using an infrared chart; if no temperature abnormal area exists, judging that no temperature abnormal defect occurs, outputting a shooting original image, and sending a signal of no temperature abnormal defect; if the abnormal temperature area exists, judging that the abnormal temperature defect occurs, sending an alarm signal of the abnormal temperature defect, and storing the position of the abnormal temperature area for the next image processing and calling;
s22: carrying out image preprocessing on the visible light image and the infrared heat image;
s23: performing wavelet multi-scale decomposition on the preprocessed visible light image by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component image obtained after the wavelet multi-scale decomposition to obtain each layer of gradient image of the visible light image;
performing wavelet multi-scale decomposition on the preprocessed infrared heat map by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component images obtained after the wavelet multi-scale decomposition to obtain each layer of gradient images of the infrared heat map;
s24: and positioning the temperature abnormal area of the gradient image of the multilayer visible light and the gradient image of the multilayer infrared heat map in a template matching mode.
Preferably, the temperature abnormality in S21 specifically includes: too high a temperature and too low a temperature.
Preferably, the preprocessing the visible light image in S22 specifically includes: graying and denoising an image; converting the obtained visible light image into a gray scale image; in order to remove noise and keep image edge detail information as much as possible, adaptive median filtering is used for denoising.
Preferably, the preprocessing of the infrared heat map at S22 specifically includes: morphological filtering and denoising; performing background suppression on the acquired infrared heat map by using morphological filtering; the morphological filtering comprises selecting structural elements, eliminating bright noise by using open operation, eliminating dark noise by using closed operation, and carrying out difference on an original image and an image obtained after processing to obtain an image with a suppressed background; in order to remove noise and keep image edge detail information as much as possible, adaptive median filtering is used for denoising.
Preferably, S24 includes the steps of:
s241: traversing the processed gradient images by using the alignment as a similarity criterion and using the sequence of firstly coarse and then fine and firstly small and then large as a matching strategy in a template matching mode, and guiding the matching search process of the fine resolution according to the result of the coarse resolution to obtain the optimal geometric transformation relation between the two images to be aligned;
s242: and calculating the corresponding position relation and range of the infrared heat map and the visible light image according to the geometric transformation relation obtained in the step S241, determining the position and area of the temperature abnormal area obtained in the step S21 in the visible light image through affine transformation, and positioning the temperature abnormal area in the visible light image.
Preferably, the specific calculation method of the alignment degree is as follows:
image IA(x, y) and image IBThe size of (x, y) is M × N, and H is defined for each gray scale N-k (k-0 to 255)A(n) and HB(n) respectively represent images IA(x, y) and image IB(x, y) the number of pixels with a gray value k, so the probability of the occurrence of the gray value k in the two images is:
for image IAEach gray level n of (x, y), now defining an image IB(x, y) relative to image IAThe mean and variance of the gray scale for the corresponding set of pixels with gray scale value n (x, y) are respectively:
with probability P of occurrence of grey value nA(n) to σ2 A,B(n) carrying out weighted average to obtain the image IADesired variance of (x, y):
by the same token can obtain
And
the interaction variance of the two images is defined as:
in the formula sigma2 AAnd σ2 BAre respectively an image IA(x, y) and image IB(x, y) variance;
therefore, the alignment is defined as:
preferably, when performing the alignment calculation, the fast traversal method of the alignment is as follows:
firstly, a 1 x 256-dimensional zero array M is establishedA、MB(ii) a Then traversing the image to obtain an image IB(x, y) a gray value k at (x, y); then M corresponding to the gray level kA(k) Carry out image IAThe gray value of (x, y) at (x, y) is accumulated to obtain an image IB(x, y) relative to image IASet M of corresponding pixels having a (x, y) grey value of kA(k) Obtaining the image I after one traversalB(x, y) relative to image IAN gray-scale corresponding pixel sets M of (x, y) gray-scale valuesA(ii) a The same way can obtain image IA(x, y) relative to image IBN gray-scale corresponding pixel sets M of (x, y) gray-scale valuesB。
Preferably, the alignment degree of the gradient images of the two images to be aligned from the 0 layer to the n layer under different translation, rotation and scaling conditions is calculated; keeping the n layers of gradient images of visible light as reference images unchanged, taking the n layers of gradient images of the infrared heat image as images to be registered, and calculating the alignment with the reference images after performing different translation, rotation and scaling transformation on the images to be registered; searching for four rigid affine transformations that maximize alignment using a search algorithmParameter (X translation X)nY translation YnAngle of rotation thetanScaling factor pn) (ii) a And then, taking n layers of output parameters as n-1 layers of input parameters, and searching by using a search algorithm in a certain range near the optimal matching position, thereby avoiding searching in the whole image range, greatly reducing useless search intervals, effectively reducing the computation load and improving the registration efficiency. Searching for four parameters of a rigid affine transformation (X translation X) that maximizes the alignmentn-1Y translation Yn-1Angle of rotation thetan-1Scaling factor pn-1n) until 0 layer (original image) registration result is output, and the registration result is used as the optimal geometric transformation relation between the two images to be registered.
The system can realize the temperature abnormal defect detection and positioning method, which comprises the following steps: the system comprises an infrared thermal imaging temperature measurement module, a visible light imaging module and an information interaction system;
the information interaction system comprises a parameter setting module, a result display module, an alarm module and an image analysis positioning module;
the infrared thermal imaging temperature measurement module collects a field infrared thermal image and transmits infrared thermal image data to the image analysis positioning module;
the visible light imaging module is used for acquiring a field visible light image and transmitting visible light image data to the image analysis positioning module;
the parameter setting module is used for setting parameters of the infrared thermal imaging temperature measurement module, setting parameters of the visible light imaging module, setting result display and setting alarm information by a user;
the result display module performs image display and related information prompt of the temperature anomaly detection positioning result;
the alarm module sends an alarm prompt to the outside according to the temperature anomaly detection positioning result;
the image analysis positioning module receives visible light image data and infrared heat image data, and transmits image text information and alarm signals of the obtained temperature anomaly detection positioning result to the result display module and the alarm module respectively through image analysis processing.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that: the method has higher efficiency of judging and positioning the temperature abnormity defects. The registration using the feature points is complicated and complex, and the registration realization efficiency is low. The camera calibration mode is used for registration, and the requirements on the position and the installation of the camera are high. The template matching is carried out by combining gradient information with the alignment degree, so that complicated operation and processing processes of extraction of the infrared heat map and the visible light image feature points and feature registration and high requirements on the position and installation of a camera are avoided; and by using multi-scale template matching, useless search intervals are greatly reduced, the operation amount is effectively reduced, and the registration efficiency is further improved. The method comprises the following specific steps:
(1) the invention uses multi-scale gradient information to match the template by combining the alignment degree, thereby avoiding the complicated operation and processing processes of characteristic point extraction and characteristic registration and the high requirements of camera position and installation;
(2) according to the invention, a fast traversal method is used when the alignment is calculated, the calculation of the pixel set corresponding to all gray levels can be completed through one-time traversal, and the problems of large calculation amount and low efficiency caused by the fact that multiple traversals are needed for multiple gray levels are avoided.
(3) The invention uses multi-scale template matching, takes the sequence of first thick, then thin, and first small and then big as a matching strategy, and guides the matching search process of the fine resolution according to the result of the coarse resolution, thereby greatly reducing useless search intervals and effectively reducing the operation amount;
(4) the invention can self-define the input temperature threshold, can self-define and identify the temperature abnormal area in the visible light image according to the input temperature threshold, and is convenient to check and analyze.
Drawings
Fig. 1 is a flowchart of a method for detecting and positioning temperature anomaly defects according to embodiment 1.
Fig. 2 is a flowchart of image processing by the image processing module.
Fig. 3 is a schematic structural diagram of the temperature anomaly defect detection and positioning system in embodiment 2.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Example 1:
the embodiment provides a method for detecting and positioning temperature anomaly defects, as shown in fig. 1, the method includes the following steps:
s1: fixing the relative positions of the infrared thermal imaging temperature measurement module and the visible light imaging module, wherein the mirror surfaces of the infrared thermal imaging temperature measurement module and the visible light imaging module are relatively parallel; the infrared thermal imaging temperature measurement module collects infrared thermal image data, and the visible light imaging module collects visible light image data. Transmitting the infrared heat map data and the visible light image data to an image analysis positioning module;
in this embodiment, the infrared thermal imaging temperature measurement module is a thermal infrared imager, and the visible light imaging module is a visible light camera.
S2: the image analysis positioning module receives the infrared heat image data and the visible light image data, and obtains temperature anomaly detection and positioning results through image analysis processing;
s3: and transmitting the image and the character information of the temperature abnormity detection positioning result to a result display module for displaying, and transmitting a temperature abnormity detection signal to an alarm module for alarming.
Further, the step S1 includes that, in order to ensure that the infrared field of view and the visible field of view coincide as much as possible, the infrared image field of view should be included in the visible image field of view.
Further, the relative parallelism described in S1 allows for a certain declination angle, and to prevent excessive rotational declination of the captured visible light image and infrared thermal image, the declination angle range is preferably: 0 to 10 degrees.
As shown in fig. 2, the image processing flow S2 of the image processing module specifically includes:
s21: and judging the temperature abnormality according to the input temperature threshold value by using the infrared chart. If no temperature abnormal area exists, judging that no temperature abnormal defect occurs, outputting a shooting original image, and sending a signal of no temperature abnormal defect; if the abnormal temperature area exists, judging that the abnormal temperature defect occurs, sending an alarm signal of the abnormal temperature defect, and storing the position of the abnormal temperature area for the next image processing and calling;
further, the temperature abnormality in S21 specifically includes: too high a temperature and too low a temperature.
S22: image pre-processing is performed on the visible light image and the infrared heat map.
Further, the preprocessing the visible light image in S22 specifically includes: graying and denoising the image, and converting the obtained visible light image into a grayscale image; in order to remove noise and keep image edge detail information as much as possible, adaptive median filtering is used for denoising.
Further, the step of preprocessing the infrared heat map at S22 specifically includes: morphological filtering and denoising; performing background suppression on the acquired infrared heat map by using morphological filtering; the morphological filtering comprises selecting a proper structural element, eliminating bright noise by using open operation, eliminating dark noise by using closed operation, and carrying out difference on an original image and an image obtained after processing to obtain an image with a suppressed background; in order to remove noise and keep image edge detail information as much as possible, adaptive median filtering is used for denoising.
S23: in order to reduce the calculation amount and improve the calculation efficiency as much as possible, wavelet transformation is used for carrying out wavelet multi-scale decomposition on the preprocessed visible light image, and then gradient extraction is carried out on low-frequency component images of all layers obtained after the wavelet multi-scale decomposition, so that gradient images of all layers of the visible light image are obtained. And performing wavelet multi-scale decomposition on the preprocessed infrared heat map by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component images obtained after the wavelet multi-scale decomposition to obtain each layer of gradient images of the infrared heat map.
Furthermore, the wavelet transform has good time-frequency localization characteristics and multi-scale analysis capability, and original image detail information can be kept as much as possible during scale change.
Furthermore, the optimal wavelet multi-scale decomposition is 2-3 layers. In this embodiment, 2 layers are selected for the number of layers. The infrared heat image and the visible light image are decomposed into a two-layer wavelet pyramid with descending precision and size through wavelet transformation, and bior3.7 is selected as a wavelet base of the wavelet transformation.
It should be noted that the parameters such as the number of decomposition layers and the wavelet basis for performing wavelet multi-scale decomposition on the infrared image and the visible light image provided in this embodiment are not limited herein, and those skilled in the art can select and use the parameters according to actual situations.
S24: and carrying out temperature abnormal region positioning on the gradient images of the multilayer visible light and the gradient images of the multilayer infrared heat map through template matching.
S241: and traversing the gradient images obtained by processing by using the alignment as a similarity criterion and using the sequence of firstly coarse and then fine and firstly small and then large as a matching strategy in a template matching mode, and guiding the matching search process of the fine resolution according to the result of the coarse resolution to obtain the optimal geometric transformation relation between the two images to be aligned.
Further, the alignment degree indicates that the gray value of each gray value of one image corresponds to the gray value of the other image at the pixel position, and is mathematically embodied as the smallest variance.
Further, a specific calculation method for calculating the alignment degree is as follows:
image IA(x, y) and image IBThe size of (x, y) is M × N, and H is defined for each gray scale N-k (k-0 to 255)A(n) and HB(n) respectively represent images IA(x, y) and image IB(x, y) the number of pixels with a gray value k, so the probability of the occurrence of the gray value k in the two images is:
for image IAEach gray level n of (x, y), now defining an image IB(x, y) relative to image IAThe mean and variance of the gray scale for the corresponding set of pixels with gray scale value n (x, y) are respectively:
with probability P of occurrence of grey value nA(n) to σ2 A,B(n) carrying out weighted average to obtain the image IADesired variance of (x, y):
by the same token can obtain
And
the interaction variance of the two images is defined as:
in the formula sigma2 AAnd σ2 BAre respectively an image IA(x, y) and image IBVariance of (x, y).
The interaction variance reflects the stability of the two image gradient images relative to each other, and it can be seen that the more similar the contents of the two images are, the smaller the interaction variance is. For convenience of description, the alignment is defined as:
further, in the fast traversal method, when the alignment degree is calculated, the most complicated and time-consuming calculation is as follows: finding an image IB(x, y) relative to image IAAnd (x, y) a set of pixels corresponding to n gray levels of the gray scale value. The general operation needs to perform n-time traversal acquisition of n gray levels. A method for obtaining a set of pixels corresponding to all gray levels through one traversal is proposed. Firstly, a zero array M with 1 multiplied 256 dimensions (corresponding to the gray scale of 0-255) is establishedA、MB(ii) a Then traversing the image to obtain an image IB(x, y) a gray value k at (x, y); then M corresponding to the gray level kA(k) Carry out image IAThe gray value of (x, y) at (x, y) is accumulated to obtain an image IB(x, y) relative to image IASet M of corresponding pixels having a (x, y) grey value of kA(k) Obtaining the image I after one traversalB(x, y) relative to image IAN gray-scale corresponding pixel sets M of (x, y) gray-scale valuesA(ii) a The same way can obtain image IA(x, y) relative to image IBN gray-scale corresponding pixel sets M of (x, y) gray-scale valuesB。
Further, calculating the alignment degree of the gradient images of the two images to be aligned from the 0 layer (original image) to the 2 layers under different translation, rotation and scaling conditions; and taking the visible light 2-layer gradient image as a reference image to be kept unchanged, taking the infrared heat image 2-layer gradient image as an image to be registered, performing different translation, rotation and scaling transformation on the image to be registered, comparing the image to be registered with the reference image, and calculating the alignment degree. Selecting Powell search algorithm to search four parameters (X translation X) of rigid affine transformation corresponding to the maximum alignment degree2Y translation Y2Angle of rotation theta2Scaling factor p2). And then, according to the 2-layer output parameters as the 1-layer input parameters, searching is carried out in a certain range near the optimal matching position by using a Powell search algorithm, so that searching in the full-image range is avoided, useless search intervals are greatly reduced, the calculation amount is effectively reduced, and the registration efficiency is improved. Searching for a stiffness that maximizes alignmentFour parameters of affine transformation (X translation X)1Y translation Y1Angle of rotation theta1Scaling factor p1) And outputting a registration result of 0 layer as the optimal geometric transformation relation between the two images to be registered.
It should be noted that the selection of the template matching search algorithm provided in this embodiment is not unique, and is not specifically limited herein, and those skilled in the art may select and use the algorithm according to actual situations.
S242: according to the geometric transformation relation obtained in the step S241, the corresponding position relation and range of the infrared heat map and the visible light image can be calculated, the position and the area of the temperature abnormal area obtained in the step S21 in the visible light image are determined through affine transformation, and the temperature abnormal area is positioned in the visible light image.
Example 2:
the present embodiment provides a temperature anomaly defect detection and positioning system, as shown in fig. 3, the system includes: the system comprises an infrared thermal imaging temperature measurement module, a visible light imaging module and an information interaction system, wherein the information interaction system comprises a parameter setting module, a result display module, an alarm module and an image analysis positioning module; the infrared thermal imaging temperature measurement module collects a field infrared thermal image and transmits infrared thermal image data to the image analysis positioning module; the visible light imaging module is used for acquiring a field visible light image and transmitting visible light image data to the image analysis positioning module; the parameter setting module is used for setting parameters of the infrared thermal imaging temperature measurement module, setting parameters of the visible light imaging module, setting result display and setting alarm information by a user; the result display module performs image display and related information prompt of the temperature anomaly detection positioning result; the alarm module sends an alarm prompt to the outside according to the temperature anomaly detection positioning result; the image analysis positioning module receives visible light image data and infrared heat image data, and transmits image text information and alarm signals of the obtained temperature anomaly detection positioning result to the result display module and the alarm module respectively through image analysis processing.
In this embodiment, the infrared thermal imaging temperature measurement module is a thermal infrared imager, and the visible light imaging module is a visible light camera.
According to the number of detection points and the environment complexity, the inspection robot, the fixed cloud platform and the handheld binocular vision equipment can be used for image acquisition as required.
The terms describing positional relationships in the drawings are for illustrative purposes only and are not to be construed as limiting the patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (10)
1. A method for detecting and positioning temperature anomaly defects is characterized by comprising the following steps:
s1: fixing the relative positions of the infrared thermal imaging temperature measurement module and the visible light imaging module, wherein the mirror surfaces of the infrared thermal imaging temperature measurement module and the visible light imaging module are relatively parallel; the infrared thermal imaging temperature measurement module collects infrared thermal image data, and the visible light imaging module collects visible light image data; transmitting the infrared heat map data and the visible light image data to an image analysis positioning module;
s2: the image analysis positioning module receives the infrared heat image data and the visible light image data, and obtains temperature anomaly detection and positioning results through image analysis processing.
2. The method for detecting and locating the temperature anomaly defect according to claim 1, further comprising S3: and transmitting the image and the character information of the temperature abnormity detection and positioning result to a result display module for displaying, and transmitting a temperature abnormity detection signal to an alarm module for alarming.
3. The method for detecting and locating the temperature anomaly defect according to the claim 1 or 2, wherein S2 comprises the following steps:
s21: judging temperature abnormality according to an input set temperature threshold value by using an infrared chart; if no temperature abnormal area exists, judging that no temperature abnormal defect occurs, outputting a shooting original image, and sending a signal of no temperature abnormal defect; if the abnormal temperature area exists, judging that the abnormal temperature defect occurs, sending an alarm signal of the abnormal temperature defect, and storing the position of the abnormal temperature area for the next image processing and calling;
s22: carrying out image preprocessing on the visible light image and the infrared heat image;
s23: performing wavelet multi-scale decomposition on the preprocessed visible light image by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component image obtained after the wavelet multi-scale decomposition to obtain each layer of gradient image of the visible light image;
performing wavelet multi-scale decomposition on the preprocessed infrared heat map by using wavelet transformation, and performing gradient extraction on each layer of low-frequency component images obtained after the wavelet multi-scale decomposition to obtain each layer of gradient images of the infrared heat map;
s24: and positioning the temperature abnormal area of the gradient image of the multilayer visible light and the gradient image of the multilayer infrared heat map in a template matching mode.
4. The method for detecting and positioning temperature anomaly defects according to claim 3, wherein the step of preprocessing the visible light image in S22 specifically comprises the steps of: graying and denoising an image; converting the obtained visible light image into a gray scale image; in order to remove noise and keep image edge detail information as much as possible, adaptive median filtering is used for denoising.
5. The method for detecting and locating the temperature anomaly defect according to claim 4, wherein the preprocessing of the infrared heat map at S22 specifically comprises: morphological filtering and denoising; performing background suppression on the acquired infrared heat map by using morphological filtering; the morphological filtering comprises selecting structural elements, eliminating bright noise by using open operation, eliminating dark noise by using closed operation, and carrying out difference on an original image and an image obtained after processing to obtain an image with a suppressed background; in order to remove noise and keep image edge detail information as much as possible, adaptive median filtering is used for denoising.
6. The method for detecting and locating the temperature anomaly defect according to the claim 4 or 5, wherein the S24 comprises the following steps:
s241: traversing the processed gradient images by using the alignment as a similarity criterion and using the sequence of firstly coarse and then fine and firstly small and then large as a matching strategy in a template matching mode, and guiding the matching search process of the fine resolution according to the result of the coarse resolution to obtain the optimal geometric transformation relation between the two images to be aligned;
s242: and calculating the corresponding position relation and range of the infrared heat map and the visible light image according to the geometric transformation relation obtained in the step S241, determining the position and area of the temperature abnormal area obtained in the step S21 in the visible light image through affine transformation, and positioning the temperature abnormal area in the visible light image.
7. The method for detecting and positioning the temperature anomaly defect according to claim 6, wherein the specific calculation method of the alignment degree is as follows:
image IA(x, y) and image IBThe size of (x, y) is M × N, and H is defined for each gray scale N-k (k-0 to 255)A(n) and HB(n) respectively represent images IA(x, y) and image IB(x, y) the number of pixels with a gray value k, so the probability of the occurrence of the gray value k in the two images is:
for image IAEach gray level n of (x, y), now defining an image IB(x, y) relative to image IAThe mean and variance of the gray scale for the corresponding set of pixels with gray scale value n (x, y) are respectively:
with probability P of occurrence of grey value nA(n) to σ2 A,B(n) carrying out weighted average to obtain the image IADesired variance of (x, y):
by the same token can obtain
And
the interaction variance of the two images is defined as:
in the formula sigma2 AAnd σ2 BAre respectively an image IA(x, y) and image IB(x, y) variance;
therefore, the alignment is defined as:
8. the method for detecting and positioning the temperature anomaly defect according to claim 7, wherein when the alignment calculation is performed, the method for rapidly traversing the alignment comprises the following steps:
firstly, a 1 x 256-dimensional zero array M is establishedA、MB(ii) a Then traversing the image to obtain an image IB(x, y) a gray value k at (x, y); then M corresponding to the gray level kA(k) Carry out image IAThe gray value of (x, y) at (x, y) is accumulated to obtain an image IB(x, y) relative to image IASet M of corresponding pixels having a (x, y) grey value of kA(k) Obtaining the image I after one traversalB(x, y) relative to image IAN gray-scale corresponding pixel sets M of (x, y) gray-scale valuesA(ii) a The same way can obtain image IA(x, y) relative to image IBN gray-scale corresponding pixel sets M of (x, y) gray-scale valuesB。
9. The method for detecting and positioning the temperature anomaly defects according to claim 8, wherein the alignment degrees of the gradient images of 0-n layers of graphs to be aligned under different translation, rotation and scaling conditions are calculated; keeping the n layers of gradient images of visible light as reference images unchanged, taking the n layers of gradient images of the infrared heat image as images to be registered, and performing alignment calculation on the images to be registered and the reference images after performing different translation, rotation and scaling transformation on the images to be registered; searching for four parameters of rigid affine transformation, namely X translation X, which maximizes the alignment using a search algorithmnY translation YnAngle of rotation thetanScaling factor pn(ii) a And then, taking n layers of output parameters as n-1 layers of input parameters, and searching by using a search algorithm in a certain range near the optimal matching position, thereby avoiding searching in the whole image range, greatly reducing useless search intervals, effectively reducing the computation load and improving the registration efficiency. Searching for four parameters of a rigid affine transformation that maximizes the alignment, i.e. X translation Xn-1Y translation Yn-1Angle of rotation thetan-1Scaling factor pn-1And n, until 0 layer of registration result is output, and the registration result is used as the optimal geometric transformation relation between the two images to be registered.
10. A system for applying the method for detecting and locating the temperature anomaly defect according to any one of claims 1 to 9, wherein the system comprises:
the system comprises an infrared thermal imaging temperature measurement module, a visible light imaging module and an information interaction system;
the information interaction system comprises a parameter setting module, a result display module, an alarm module and an image analysis positioning module;
the infrared thermal imaging temperature measurement module collects a field infrared thermal image and transmits infrared thermal image data to the image analysis positioning module;
the visible light imaging module is used for acquiring a field visible light image and transmitting visible light image data to the image analysis positioning module;
the parameter setting module is used for setting parameters of the infrared thermal imaging temperature measurement module, setting parameters of the visible light imaging module, setting result display and setting alarm information by a user;
the result display module performs image display and related information prompt of the temperature anomaly detection positioning result;
the alarm module sends an alarm prompt to the outside according to the temperature anomaly detection positioning result;
the image analysis positioning module receives visible light image data and infrared heat image data, and transmits image text information and alarm signals of the obtained temperature anomaly detection positioning result to the result display module and the alarm module respectively through image analysis processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911244645.9A CN110942458B (en) | 2019-12-06 | 2019-12-06 | Temperature anomaly defect detection and positioning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911244645.9A CN110942458B (en) | 2019-12-06 | 2019-12-06 | Temperature anomaly defect detection and positioning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110942458A true CN110942458A (en) | 2020-03-31 |
| CN110942458B CN110942458B (en) | 2023-05-16 |
Family
ID=69910082
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911244645.9A Active CN110942458B (en) | 2019-12-06 | 2019-12-06 | Temperature anomaly defect detection and positioning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110942458B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111242144A (en) * | 2020-04-26 | 2020-06-05 | 北京邮电大学 | Method and device for detecting abnormality of power grid equipment |
| CN111553910A (en) * | 2020-05-06 | 2020-08-18 | 哈尔滨工业大学 | Infrared image-based thermal fault detection method for electrical equipment of large experimental device |
| CN111739069A (en) * | 2020-05-22 | 2020-10-02 | 北京百度网讯科技有限公司 | Image registration method and device, electronic equipment and readable storage medium |
| CN111795745A (en) * | 2020-05-29 | 2020-10-20 | 红鼎互联(广州)信息科技有限公司 | Temperature calibration processing method for infrared image |
| CN112539843A (en) * | 2020-12-11 | 2021-03-23 | 平安科技（深圳）有限公司 | Method and device for detecting temperature and computer equipment |
| CN112540635A (en) * | 2020-12-03 | 2021-03-23 | 华润三九（枣庄）药业有限公司 | Traditional chinese medicine production intelligence quality control system based on artificial intelligence |
| CN113674271A (en) * | 2021-09-06 | 2021-11-19 | 广东康德威电气股份有限公司 | Transformer monitoring system based on cloud computing |
| CN114577405A (en) * | 2022-02-28 | 2022-06-03 | 辽宁石油化工大学 | Urban underground sewage pipeline defect detection device based on infrared imaging technology |
| CN115375588A (en) * | 2022-10-25 | 2022-11-22 | 山东旗胜电气股份有限公司 | Power grid transformer fault identification method based on infrared imaging |
| CN115661016A (en) * | 2022-12-08 | 2023-01-31 | 瑞纳智能设备股份有限公司 | Fault monitoring method and system of heat exchange station and embedded image diagnosis control platform |
| CN116739994A (en) * | 2023-05-18 | 2023-09-12 | 山东新宝龙工业科技有限公司 | Digital conveyer belt graphic data monitoring system with early warning function |
| CN111739069B (en) * | 2020-05-22 | 2024-04-26 | 北京百度网讯科技有限公司 | Image registration method, device, electronic equipment and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025648A (en) * | 2017-03-20 | 2017-08-08 | 中国人民解放军空军工程大学 | A kind of board failure infrared image automatic testing method |
| CN107253485A (en) * | 2017-05-16 | 2017-10-17 | 北京交通大学 | Foreign matter invades detection method and foreign matter intrusion detection means |
| CN109410159A (en) * | 2018-09-11 | 2019-03-01 | 上海创客科技有限公司 | Binocular visible light and infrared thermal imaging complex imaging system, method and medium |
-
2019
- 2019-12-06 CN CN201911244645.9A patent/CN110942458B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025648A (en) * | 2017-03-20 | 2017-08-08 | 中国人民解放军空军工程大学 | A kind of board failure infrared image automatic testing method |
| CN107253485A (en) * | 2017-05-16 | 2017-10-17 | 北京交通大学 | Foreign matter invades detection method and foreign matter intrusion detection means |
| CN109410159A (en) * | 2018-09-11 | 2019-03-01 | 上海创客科技有限公司 | Binocular visible light and infrared thermal imaging complex imaging system, method and medium |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111242144A (en) * | 2020-04-26 | 2020-06-05 | 北京邮电大学 | Method and device for detecting abnormality of power grid equipment |
| CN111553910A (en) * | 2020-05-06 | 2020-08-18 | 哈尔滨工业大学 | Infrared image-based thermal fault detection method for electrical equipment of large experimental device |
| CN111553910B (en) * | 2020-05-06 | 2022-03-25 | 哈尔滨工业大学 | Infrared image-based thermal fault detection method for electrical equipment of large experimental device |
| CN111739069A (en) * | 2020-05-22 | 2020-10-02 | 北京百度网讯科技有限公司 | Image registration method and device, electronic equipment and readable storage medium |
| CN111739069B (en) * | 2020-05-22 | 2024-04-26 | 北京百度网讯科技有限公司 | Image registration method, device, electronic equipment and readable storage medium |
| CN111795745A (en) * | 2020-05-29 | 2020-10-20 | 红鼎互联(广州)信息科技有限公司 | Temperature calibration processing method for infrared image |
| CN112540635A (en) * | 2020-12-03 | 2021-03-23 | 华润三九（枣庄）药业有限公司 | Traditional chinese medicine production intelligence quality control system based on artificial intelligence |
| CN112539843B (en) * | 2020-12-11 | 2023-06-02 | 平安科技（深圳）有限公司 | Method and device for detecting temperature and computer equipment |
| CN112539843A (en) * | 2020-12-11 | 2021-03-23 | 平安科技（深圳）有限公司 | Method and device for detecting temperature and computer equipment |
| CN113674271A (en) * | 2021-09-06 | 2021-11-19 | 广东康德威电气股份有限公司 | Transformer monitoring system based on cloud computing |
| CN114577405A (en) * | 2022-02-28 | 2022-06-03 | 辽宁石油化工大学 | Urban underground sewage pipeline defect detection device based on infrared imaging technology |
| CN115375588B (en) * | 2022-10-25 | 2023-02-07 | 山东旗胜电气股份有限公司 | Power grid transformer fault identification method based on infrared imaging |
| CN115375588A (en) * | 2022-10-25 | 2022-11-22 | 山东旗胜电气股份有限公司 | Power grid transformer fault identification method based on infrared imaging |
| CN115661016B (en) * | 2022-12-08 | 2023-04-18 | 瑞纳智能设备股份有限公司 | Fault monitoring method and system of heat exchange station and embedded image diagnosis control platform |
| CN115661016A (en) * | 2022-12-08 | 2023-01-31 | 瑞纳智能设备股份有限公司 | Fault monitoring method and system of heat exchange station and embedded image diagnosis control platform |
| CN116739994A (en) * | 2023-05-18 | 2023-09-12 | 山东新宝龙工业科技有限公司 | Digital conveyer belt graphic data monitoring system with early warning function |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110942458B (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110942458A (en) | Temperature anomaly defect detection and positioning method and system | |
| CN109870461B (en) | Electronic components quality detection system | |
| US8797387B2 (en) | Self calibrating stereo camera | |
| US9007490B1 (en) | Approaches for creating high quality images | |
| US20150262346A1 (en) | Image processing apparatus, image processing method, and image processing program | |
| CN109919933B (en) | VR device, picture detection method and device thereof, and computer-readable storage medium | |
| JP4533836B2 (en) | Fluctuating region detection apparatus and method | |
| CN110322485B (en) | Rapid image registration method of heterogeneous multi-camera imaging system | |
| CN111563896B (en) | Image processing method for detecting abnormality of overhead line system | |
| CN110634137A (en) | Bridge deformation monitoring method, device and equipment based on visual perception | |
| CN112927223A (en) | Glass curtain wall detection method based on infrared thermal imager | |
| CN113095358A (en) | Image fusion method and system | |
| CN114372948A (en) | Image fusion method, device and storage medium | |
| JP5600954B2 (en) | Inspection system, method and program | |
| TWI496115B (en) | Video frame stabilization method for the moving camera | |
| CN114359149A (en) | Dam bank dangerous case video detection method and system based on real-time image edge enhancement | |
| CA3206206A1 (en) | Device and method for correspondence analysis in images | |
| JP2010217984A (en) | Image detector and image detection method | |
| CN114119652A (en) | Method and device for three-dimensional reconstruction and electronic equipment | |
| JP6080424B2 (en) | Corresponding point search device, program thereof, and camera parameter estimation device | |
| CN113008470B (en) | Gas leakage detection device and gas leakage detection method | |
| JP2022057784A (en) | Imaging apparatus, imaging system, and imaging method | |
| CN113591705B (en) | Inspection robot instrument identification system and method and storage medium | |
| CN110750757A (en) | Image jitter amount calculation method based on gray scale linear modeling and pyramid decomposition | |
| CN109840894A (en) | Disparity map refine method, apparatus and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231025 Address after: Building B, 3-1B, Phase II, Diejin Industrial Zone, Jinping District, Shantou, Guangdong, 515000 Patentee after: SHANTOU A-PLUX OPTOELECTRONIC TECHNOLOGY CO.,LTD. Address before: 515063 No. 243, University Road, Shantou, Guangdong Patentee before: SHANTOU University |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240124 Address after: Building 3, 6th Floor, Financial Center, Dongshan Street Office, Nankang District, Ganzhou City, Jiangxi Province, 341000 Patentee after: Jiangxi Xinkang Technology Co.,Ltd. Country or region after: China Address before: Building B, 3-1B, Phase II, Diejin Industrial Zone, Jinping District, Shantou, Guangdong, 515000 Patentee before: SHANTOU A-PLUX OPTOELECTRONIC TECHNOLOGY CO.,LTD. Country or region before: China |
|
| TR01 | Transfer of patent right |