CN112967204A - Noise reduction processing method and system for thermal imaging and electronic equipment - Google Patents
Noise reduction processing method and system for thermal imaging and electronic equipment Download PDFInfo
- Publication number
- CN112967204A CN112967204A CN202110309229.3A CN202110309229A CN112967204A CN 112967204 A CN112967204 A CN 112967204A CN 202110309229 A CN202110309229 A CN 202110309229A CN 112967204 A CN112967204 A CN 112967204A
- Authority
- CN
- China
- Prior art keywords
- array data
- thermal imaging
- face
- temperature array
- noise reduction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001931 thermography Methods 0.000 title claims abstract description 153
- 230000009467 reduction Effects 0.000 title claims abstract description 122
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 102
- 238000000034 method Methods 0.000 claims description 51
- 238000001914 filtration Methods 0.000 claims description 38
- 230000008569 process Effects 0.000 claims description 24
- 238000001514 detection method Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 238000011946 reduction process Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 5
- 230000002146 bilateral effect Effects 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000009499 grossing Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 210000000887 face Anatomy 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 210000001061 forehead Anatomy 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/0022—Radiation pyrometry, e.g. infrared or optical thermometry for sensing the radiation of moving bodies
- G01J5/0025—Living bodies
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J2005/0077—Imaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a noise reduction processing method and system for thermal imaging and electronic equipment, wherein a face region in an RGB image is identified by acquiring RGB image and thermal imaging temperature array data of the same region at the same time, the face temperature array data is determined according to the position of the face region in the RGB image and the position corresponding relation of the face temperature array data in the thermal imaging temperature array data, and the noise reduction processing is carried out on non-face temperature array data, so that the independent noise reduction processing can be carried out on the non-face temperature array data in the thermal imaging temperature array data, and the situation that a large amount of face temperature details in the face temperature array data are erased due to the fact that the face temperature array data adopts a noise reduction processing method which is the same as the non-face temperature array data is avoided.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a thermal imaging noise reduction processing method and system and electronic equipment.
Background
All objects in nature can radiate infrared rays, the radiation energy of the infrared rays is in direct proportion to the fourth power of the temperature of the infrared rays, and the radiated wavelength is in inverse proportion to the temperature of the infrared rays. The thermal imaging technology (infrared thermal imaging technology) is a thermal image which is processed by a system according to the level of the detected radiation energy of an object and is converted into a target object, and the target object is displayed in gray scale or pseudo color and can be used for monitoring the temperature distribution condition of the target object.
In the existing thermal imaging technology, noise reduction processing needs to be performed on the acquired thermal imaging temperature array data, and noise in an image needs to be erased, so that smoothness of a thermal imaging picture is improved. However, in erasing the noise in the image, the temperature information of the human face in the thermal imaging picture is also erased in a large amount.
Therefore, the prior art is in need of improvement.
Disclosure of Invention
In view of this, the present application provides a method and a system for denoising in thermal imaging, and an electronic device, so as to solve the problem that the existing denoising process can largely erase the temperature information of the face in the thermal imaging picture.
In one aspect, a method for noise reduction processing of thermal imaging is provided, including:
acquiring RGB images and thermal imaging temperature array data of the same area at the same time;
carrying out face detection on the RGB image to obtain a face area in the RGB image;
determining face temperature array data in the thermal imaging temperature array data through the face area in the RGB image according to the position corresponding relation between the face area in the RGB image and the position corresponding relation of the face temperature array data in the thermal imaging temperature array data;
carrying out first noise reduction processing on non-human face temperature array data in the thermal imaging temperature array data to obtain thermal imaging temperature array data subjected to noise reduction processing;
and mapping the thermal imaging temperature array data subjected to the noise reduction processing into an image for displaying.
In an embodiment of the present invention, before obtaining the thermal imaging temperature array data after the noise reduction processing, the method further includes:
and carrying out second noise reduction processing on the face temperature array data, wherein the signal-to-noise ratio of the non-face temperature array data subjected to the first noise reduction processing is greater than the signal-to-noise ratio of the face temperature array data subjected to the second noise reduction processing.
In one embodiment, the noise-reduced thermographic temperature array data is mapped to a color image.
In one embodiment, the noise reduction methods adopted by the first noise reduction processing and the second noise reduction processing are one or more of mean value filtering noise reduction processing, kalman filtering noise reduction processing, bilateral filtering noise reduction and time sequence filtering noise reduction processing.
In one embodiment, the formula of the mean filtering noise reduction process is as follows:
i. j represents an abscissa and an ordinate in the thermal imaging temperature array data respectively, a' (i, j) represents temperature data corresponding to coordinates (i, j) after mean filtering and noise reduction processing, a (i, j) represents temperature data corresponding to coordinates (i, j) in the thermal imaging temperature array data, a (i + m, j + n) represents temperature data corresponding to coordinates (i + m, j + n) in the thermal imaging temperature array data, r is a filtering radius, and gamma is a preset adjustment coefficient, wherein the value of gamma is greater than or equal to 0 and less than or equal to 1, and the value of gamma adopted by the first noise reduction processing is greater than the value of gamma adopted by the second noise reduction processing.
In one embodiment, the first noise reduction process uses a larger filter radius than the second noise reduction process.
In one embodiment, the calculation formula of the filter radius in the first denoising process is as follows:
In one embodiment, the face detection on the RGB image includes:
performing multi-scale transformation on the RGB image to construct an image pyramid;
processing the image pyramid by adopting a convolutional neural network, and identifying to obtain a plurality of face windows and probability values corresponding to the face windows;
inversely transforming the face window with the probability value larger than the preset probability value to the RGB image to form a plurality of face frames;
and eliminating the crossed and repeated face frames through a non-maximum suppression algorithm to obtain a face area in the RGB image.
In one embodiment, determining the face temperature array data in the thermal imaging temperature array data from the face region in the RGB image comprises:
acquiring coordinates of a face area in the RGB image according to the face area in the RGB image;
converting the coordinates of the face area in the RGB image to obtain the coordinates of the face temperature array data in the thermal imaging temperature array data according to the corresponding relation between the coordinates of the face area in the RGB image and the coordinates of the face temperature array data in the thermal imaging temperature array data;
and determining the face temperature array data according to the coordinates of the face temperature array data.
There is provided an electronic device comprising a memory and a processor, the memory storing a computer program executable by the processor for implementing the steps of the method of noise reduction processing for thermal imaging as above.
The system comprises a data acquisition module, a face detection module, a conversion module and a noise reduction module, wherein the data acquisition module is connected with the face detection module, the face detection module is connected with the conversion module, and the conversion module is connected with the noise reduction module;
the data acquisition module is used for acquiring RGB images and thermal imaging temperature array data at the same moment;
the face detection module is used for carrying out face detection on the RGB image to acquire a face area in the RGB image;
the conversion module is used for determining the face temperature array data in the thermal imaging temperature array data through the face area in the RGB image according to the corresponding relation between the position of the face area in the RGB image and the position of the face temperature array data in the thermal imaging temperature array data;
and the noise reduction module is used for carrying out first noise reduction processing on the non-human face temperature array data in the thermal imaging temperature array data.
According to the noise reduction processing method for the thermal imaging, the face area in the RGB image is identified, the face temperature array data is determined according to the position of the face area in the RGB image and the position corresponding relation of the face temperature array data in the thermal imaging temperature array data, and noise reduction processing is performed on the non-face temperature array data, so that independent noise reduction processing can be performed on the non-face temperature array data in the thermal imaging temperature array data, and the situation that a large amount of face temperature details in the face temperature array data are erased due to the fact that the face temperature array data adopt the same noise reduction processing method as the non-face temperature array data is avoided.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of a prior art thermal imaging display method;
FIG. 2 is a flow chart of a method for denoising in thermal imaging according to an embodiment of the present disclosure;
FIG. 3 is a schematic view of an application scenario of a noise reduction processing method for thermal imaging according to an embodiment of the present application;
FIG. 4 is a schematic view illustrating a process of detecting a face in an RGB image according to an embodiment of the present application;
FIG. 5 is a flowchart illustrating a method for denoising in thermal imaging according to an embodiment of the present application;
FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a thermal imaging noise reduction processing system according to an embodiment of the present application.
Detailed Description
In the embodiment, the independent noise reduction processing of the non-human face temperature data is realized by distinguishing the human face temperature data and the non-human face temperature data in the thermal imaging temperature array data. The technical solutions in the embodiments of the present application are clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. The following embodiments and their technical features may be combined with each other without conflict.
In one aspect, a method for denoising in thermal imaging is provided, referring to fig. 2, including steps 100 to 400.
and 500, mapping the thermal imaging temperature array data subjected to the noise reduction processing into an image for displaying.
According to the noise reduction processing method for the thermal imaging, the face area in the RGB image is identified, the face temperature array data is determined according to the position of the face area in the RGB image and the position corresponding relation of the face temperature array data in the thermal imaging temperature array data, and noise reduction processing is performed on the non-face temperature array data, so that independent noise reduction processing can be performed on the non-face temperature array data in the thermal imaging temperature array data, and the situation that a large amount of face temperature details in the face temperature array data are erased due to the fact that the face temperature array data adopt the same noise reduction processing method as the non-face temperature array data is avoided.
In step 100, the RGB image and the thermal imaging temperature array data are simultaneously acquired from the same region, where the RGB image may be acquired by an RGB camera, and the temperature array data may be acquired by a thermal imaging detector (thermal imaging detector). It should be noted that the temperature array data is a temperature data matrix in which a plurality of temperature data are arranged, where the arrangement of the temperature data corresponds to the temperature distribution of the actual collection area.
The RGB image and the thermal imaging temperature array data of the same area refer to RGB image and thermal imaging temperature array data acquired from the same area, for example, referring to fig. 3, the shooting range of the RGB image 430 is an area of a hall entrance 450, and the thermal imaging detector 440 also acquires temperature array data of the hall entrance area.
In one embodiment, the RGB camera and the thermal imaging detector are disposed adjacent to each other and oriented in the same direction, so as to acquire RGB images and thermal imaging image temperature array data of the same region at the same angle.
In step 200, a face region in the RGB image is obtained by performing face detection on the RGB image. In one embodiment, referring to fig. 4, detecting a face in an RGB image by using an artificial intelligence face recognition algorithm includes:
and step 204, eliminating the crossed and repeated face frames through a non-maximum suppression algorithm to obtain a face area in the RGB image.
The embodiment adopts an artificial intelligence face recognition algorithm, can accurately detect faces with different sizes in the RGB image in real time, and obtains the face area in the RGB image.
In step 201, since the sizes of the faces in the RGB Image data are different, an Image Pyramid (Image Pyramid) needs to be used for multi-scale transformation, so that the detection of the faces with different sizes in the RGB Image data can be realized.
In step 202, the convolutional neural network is composed of convolutional layers, pooling layers, and fully-connected layers, for example, the convolutional neural network includes 5 convolutional layers, 3 pooling layers, and 3 fully-connected layers. The convolution layer is used for extracting high-dimensional features of the RGB image. The activation function ReLU in the convolutional layer performs a non-linear mapping on the output result. The pooling layer, also known as undersampling or downsampling, is mainly used for reducing feature size, compressing the number of data and parameters, reducing overfitting, improving the fault tolerance of the model, and enabling the model to be fitted more quickly and better towards the optimal direction. In particular, the convolutional neural network may be an AlexNet convolutional neural network.
In step 203, the preset probability value is used for filtering the face window with a lower probability value, and the face window with a higher probability value is reserved, so that the calculation amount of subsequent processing can be effectively reduced. Optionally, the preset probability value is 95% or 98%.
In this embodiment, the face window satisfying the preset probability value is inversely transformed to the RGB image, and a plurality of frame-shaped regions, that is, face frames, are formed on the RGB image, where the region covered by the face frame is the corresponding face region on the RGB image. Alternatively, the shape of the face box may be the outline of the target face.
In the RGB image, a plurality of frame areas are face areas, but a plurality of overlapped frame areas may appear on the same face, and step 204 eliminates the cross-over window and finds a better face area through a non-maximum suppression algorithm.
In step 300, the face temperature array data is array data composed of face temperatures corresponding to the thermal imaging temperature array data. It will be appreciated that the face temperature array data is temperature data for one or more regions of the thermographic temperature array data, for example the location of the face temperature array data is in a central region of the thermographic temperature array data.
Because the RGB image and the thermal imaging frame temperature array are data acquired in the same region at the same time, that is, the spatial position of the face in the region is determined, the position of the face region in the RGB image and the position of the face temperature array data in the thermal imaging temperature array data have a corresponding relationship or the same positional relationship. For example, the face region is located at the upper left corner of the RGB image, for example, the center of the face region is 1/4 upper side lengths away from the left side of the RGB image, and 2/3 left side lengths away from the upper side of the RGB image; similarly, the face temperature array data is also located in the upper left corner of the thermal imaging temperature array data, and the center of the face temperature array data is 1/4 lower side lengths away from the left side of the thermal imaging temperature array data and 2/3 left side lengths away from the lower side of the RGB image. Therefore, the human face temperature array data in the thermal imaging temperature array data can be determined through the human face area in the RGB image.
In one embodiment, the coordinate system is used to represent the position of the face region in the RGB image and the position of the face temperature array data in the thermal imaging temperature array data. The coordinates of the RGB image can be established by taking the number of pixels as a unit, for example, the RGB image (10,10) represents pixels from the left to the right, the 10 th column, the bottom to the 10 th row; the coordinates of the thermographic temperature array data may be established by taking the number of temperatures as a unit, e.g., the thermographic temperature array data (10,10) represents the temperature data from the left to the right, column 10, bottom to top, row 10 in the array.
Optionally, the determining the face temperature array data in the thermal imaging temperature array data according to the position of the face region in the RGB image includes:
301, acquiring coordinates of a face area in the RGB image according to the face area in the RGB image;
step 302, converting the coordinates of the face area in the RGB image into the coordinates of the face temperature array data in the thermal imaging temperature array data according to the corresponding relation between the face area in the RGB image and the coordinates of the face temperature array data in the thermal imaging temperature array data;
step 303, determining the face temperature array data in the thermal imaging temperature array data according to the coordinates of the face temperature array data in the thermal imaging temperature array data.
Step 301 may establish a coordinate system according to two adjacent sides of the RGB image as coordinate axes, and obtain coordinates of the face region. A coordinate system is established based on the first column and the first row of the thermographic temperature array data as coordinate axes. In step 302, when the number of pixels in the RGB image is not equal to the number of temperature data of the thermal imaging temperature array data, a corresponding relationship may be that one pixel corresponds to a plurality of temperature data or that a plurality of pixels corresponds to one temperature data.
In step 400, the non-face temperature array data is the temperature data of the thermal imaging temperature array data excluding the face temperature array data. By performing the first noise reduction processing on the non-human face temperature array data, the image smoothness of the background area in the thermal imaging picture can be improved.
In an embodiment, referring to fig. 5, before obtaining the thermal imaging temperature array data after the noise reduction processing, the method further includes:
and 600, performing second noise reduction processing on the face temperature array data, wherein the signal-to-noise ratio of the non-face temperature array data subjected to the first noise reduction processing is greater than the signal-to-noise ratio of the face temperature array data subjected to the second noise reduction processing.
In this embodiment, by performing the second denoising processing on the face temperature array data, the smoothness of the face region in the obtained thermal imaging picture is improved, and the look and feel of the face region are improved. It should be noted that the second denoising processing performed on the face temperature array data is relatively conservative, and excessive denoising cannot be performed to obscure a large amount of temperature detail information. That is, the noise reduction parameters for making the smoothness of the picture higher are adopted in the first noise reduction processing, and the noise reduction parameters for making the smoothness of the picture lower are adopted in the second noise reduction processing, so that more face temperature details are kept as much as possible. In other words, the smoothing effect of the first noise reduction processing is better than that of the second noise reduction processing, and the signal-to-noise ratio of the non-human face temperature array data after the first noise reduction processing is smaller than that of the human face temperature array data after the second noise reduction processing.
At present, the flow of the display method of thermal imaging, please refer to fig. 1, which includes: step 10, reading thermal imaging temperature array data from a thermal imaging detector (Sensor); step 20, preprocessing the thermal imaging temperature array data to obtain gray data; step 30, mapping the gray data obtained after the preprocessing into a color image; and step 40, displaying the converted color image (namely, a thermal imaging image).
The preprocessing of the thermal imaging temperature array data is specifically to perform noise reduction processing on the temperature array data. If the noise reduction processing parameters are too conservative, the smoothing effect of the thermal imaging image is insufficient, so that the overall noise of the thermal imaging image is more, the appearance is poor, and the method has the advantages that the human face (head) can keep more detail temperature, and the temperature of each part (such as forehead) of the human face details can be conveniently and visually sensed; if the noise reduction processing parameters are too aggressive, the thermal imaging image has a better smoothing effect, the thermal imaging image has less noise and better impression, but the defect is that the temperature details at the face are blurred at the same time, and the temperature at the face details is difficult to distinguish.
In the current noise reduction processing process, the face temperature data and the non-face temperature data in the thermal imaging temperature array data cannot be distinguished, and only the unified noise reduction processing can be adopted for all the thermal imaging temperature array data.
In the embodiment, conservative noise reduction parameters are adopted for the face temperature array data, so that more face detail information is reserved; and adopting aggressive noise reduction parameters for the non-human face temperature array data (corresponding to the background area) to make the background smoother, erase background noise and improve the overall impression of the thermal imaging picture.
In step 600, a first denoising process and a second denoising process are used to denoise a non-face region and a face region respectively, so as to achieve the purpose of smoothing the image of the non-face region in a thermal imaging picture and retaining more face temperature detail information in the face region. In one embodiment, the first denoising process and the second denoising process respectively include one or more of a mean filtering denoising process, a kalman filtering denoising process, a bilateral filtering denoising process, and a time-series filtering denoising process.
The mean filtering and noise reduction processing is that a mean filter adopting a neighborhood averaging method is very suitable for removing particle noise in an image obtained by scanning. The Kalman filtering noise reduction processing is to process noisy input and observation signals on the basis of linear state space representation to obtain a system state or a real signal. The bilateral filtering denoising processing is a nonlinear filtering method, is compromise processing combining the spatial proximity and the pixel value similarity of an image, and simultaneously considers the spatial domain information and the gray level similarity to achieve the purpose of edge-preserving denoising. The time sequence filtering noise reduction processing is also called time sequence noise reduction processing, and is a nonlinear threshold processing method, the principle of which is that after wavelet transformation, the energy of a useful signal is concentrated on a few wavelet coefficients, and white noise is still dispersed on a large number of wavelet coefficients on a wavelet transformation domain, wherein the wavelet coefficient value of the useful signal is larger than that of noise with energy dispersion and smaller amplitude, so that the useful signal and the noise can be separated.
In one embodiment, the first denoising process and the second denoising process are both mean filtering denoising processes, and a formula of the mean filtering denoising process is as follows:
i. j represents an abscissa and an ordinate in the thermal imaging temperature array data respectively, a' (i, j) represents temperature data corresponding to coordinates (i, j) after mean filtering and noise reduction processing, a (i, j) represents temperature data corresponding to coordinates (i, j) in the thermal imaging temperature array data, a (i + m, j + n) represents temperature data corresponding to coordinates (i + m, j + n) in the thermal imaging temperature array data, r is a filtering radius, and gamma is a preset adjustment coefficient. Alternatively, γ is 0 or more and 1 or less.
Wherein γ used for the first noise reduction processing is larger than γ used for the second noise reduction processing. Optionally, γ used for the first noise reduction processing is greater than 0.5, γ used for the second noise reduction processing is less than 0.5, γ used for the first noise reduction processing is 0.8 or 1, and γ used for the second noise reduction processing is 0 or 0.1.
According to the embodiment, different gamma values are set according to different areas where the pixels are located, so that different noise reduction effects on different areas are achieved. Specifically, gamma used in the first noise reduction processing is larger than gamma used in the second noise reduction processing, so that the image of the non-face area can be smoother, and more face temperature details can be reserved in the face area.
In one embodiment, the first noise reduction process uses a larger filter radius than the second noise reduction process. In the mean noise reduction processing, the larger the filter radius is, the smoother the processed image is. Optionally, the calculation formula of the filter radius in the first denoising process is:
wherein i and j represent an abscissa and an ordinate in the thermal imaging temperature array data respectively, R is a filtering radius, R is a preset filtering radius, a and b are an abscissa and an ordinate corresponding to a maximum temperature value in the face temperature array data respectively, and L is the number of single-row temperature data in the thermal imaging temperature array data.
The filtering radius in the second noise reduction processing is smaller than the minimum value of the filtering radius in the first noise reduction processing. When a and b are respectively set as an abscissa and an ordinate corresponding to central temperature array data (central temperature array data) in the face temperature array data, the calculation formula of the filter radius in the first noise reduction processing and the filter radius in the second noise reduction processing is as follows:
in the first denoising process performed on the non-face region in this embodiment, the filtering radius is dynamically adjusted according to the distance from the coordinate of the maximum temperature value in the face temperature array data, where the larger the distance from the coordinate of the maximum temperature value in the face temperature array data is, the larger the filtering radius is, the larger the smoothness of the region farther from the face region in the thermal imaging picture is, and the picture impression is improved.
The noise-reduced thermographic temperature array data may be mapped into a color image in step 500.
The present embodiment can convert thermal imaging temperature array data into a color image by setting in advance the correspondence relationship between temperature and visible color. It will be appreciated that the above method may also map the post-thermographic temperature array data of the first and second noise reduction processes into a color image.
The method for denoising the thermal imaging comprises the steps of identifying a face region in an RGB image, determining face temperature array data according to the position of the face region in the RGB image and the position corresponding relation of the face temperature array data in the thermal imaging temperature array data, denoising non-face temperature array data, independently denoising the non-face temperature array data in the thermal imaging temperature array data, and denoising the face temperature array data by adopting different parameters from the non-face temperature array data, so that conservative denoising parameters can be adopted for the face temperature array data, and more face detail information is reserved; and adopting aggressive noise reduction parameters for the non-human face temperature array data (corresponding to the background area) to make the background smoother, erase background noise and improve the overall impression of the thermal imaging picture.
An embodiment of the present application further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the thermal imaging noise reduction processing method in any of the above embodiments. The electronic devices described therein may include mobile terminals such as smart thermometers, cell phones, tablets, navigation devices, wearable devices, smart bands, monitoring devices, and the like. Fig. 6 is a schematic diagram of an internal structure of an electronic device in one embodiment. As shown in fig. 6, the electronic device includes a processor, a memory, and a network interface connected by a system bus. Wherein, the processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment. The memory is used for storing data, programs and the like, and the memory stores at least one computer program which can be executed by the processor to realize the wireless network communication method suitable for the electronic device provided by the embodiment of the application. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by a processor to implement a thermal imaging noise reduction processing method provided by the above embodiments. The internal memory provides a cached execution environment for the operating system computer programs in the non-volatile storage medium. The network interface may be an ethernet card or a wireless network card, etc. for communicating with an external electronic device.
While the following description will be given taking an intelligent thermometer as an example, those skilled in the art will appreciate that the configuration according to the embodiment of the present application can be applied to a terminal for mobile purposes, in addition to being particularly used for a terminal of a fixed type.
Any reference to memory, storage, database, or other medium used herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).
One or more non-transitory readable storage media storing computer readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the method of noise reduction processing for thermal imaging as in any one of the embodiments above are also presented.
Referring to fig. 7, a thermal imaging noise reduction processing system is provided, and includes a background server 410, an electronic terminal 420, a camera 430 and a thermal imaging detector 440, where the background server 410 is in communication connection with the electronic terminal 420 and the camera 430, and the electronic terminal 420 is in communication connection with the camera 430; (ii) a
A camera 430 and a thermal imaging detector 440, which are respectively used for acquiring RGB images and thermal imaging temperature array data at the same time, and uploading the RGB images and the thermal imaging temperature array data to the background server 410;
the background server 410 is configured to perform face detection on the RGB image to obtain a face region in the RGB image; determining face temperature array data in the thermal imaging temperature array data through the face area in the RGB image according to the position corresponding relation between the face area in the RGB image and the position corresponding relation of the face temperature array data in the thermal imaging temperature array data; carrying out first noise reduction processing on non-human face temperature array data in the thermal imaging temperature array data to obtain thermal imaging temperature array data subjected to noise reduction processing;
and the electronic terminal 420 is used for converting the thermal imaging temperature array data subjected to the noise reduction processing into an image for displaying.
In some embodiments, in the noise reduction processing system for thermal imaging, the background server 410 is further configured to perform a second noise reduction processing on the face temperature array data, where a signal-to-noise ratio of the non-face temperature array data after the first noise reduction processing is greater than a signal-to-noise ratio of the face temperature array data after the second noise reduction processing.
For specific limitations of the noise reduction processing system for thermal imaging according to the embodiment of the present application, refer to the aforementioned noise reduction processing method for thermal imaging, and are not described again. The foregoing noise reduction processing method for thermal imaging may also refer to the specific definition of the noise reduction processing system for thermal imaging in the embodiment of the present application.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for noise reduction processing of thermal imaging, comprising:
acquiring RGB images and thermal imaging temperature array data of the same area at the same time;
carrying out face detection on the RGB image to obtain a face area in the RGB image;
determining the human face temperature array data in the thermal imaging temperature array data through the human face area in the RGB image according to the position corresponding relation between the human face area in the RGB image and the position corresponding relation between the human face temperature array data in the thermal imaging temperature array data;
carrying out first noise reduction processing on non-human face temperature array data in the thermal imaging temperature array data to obtain thermal imaging temperature array data subjected to noise reduction processing;
and mapping the thermal imaging temperature array data subjected to the noise reduction processing into an image for displaying.
2. The method of claim 1, wherein before obtaining the noise-reduced thermal imaging temperature array data, the method further comprises:
and carrying out second noise reduction processing on the face temperature array data, wherein the signal-to-noise ratio of the non-face temperature array data subjected to the first noise reduction processing is greater than the signal-to-noise ratio of the face temperature array data subjected to the second noise reduction processing.
3. The thermal imaging noise reduction processing method according to claim 2, wherein the first noise reduction processing and the second noise reduction processing are performed by one or more of mean value filtering noise reduction processing, kalman filtering noise reduction processing, bilateral filtering noise reduction, and time sequence filtering noise reduction processing.
4. The method of claim 3, wherein the formula of the mean filtering noise reduction process is as follows:
wherein i and j respectively represent an abscissa and an ordinate in the thermal imaging temperature array data, a' (i, j) represents temperature data corresponding to the coordinate (i, j) after the mean filtering denoising process, a (i, j) represents temperature data corresponding to the coordinate (i, j) in the thermal imaging temperature array data, a (i + m, j + n) represents temperature data corresponding to the coordinate (i + m, j + n) in the thermal imaging temperature array data, r is a filtering radius, and γ is a preset adjustment coefficient, wherein a value of γ is greater than or equal to 0 and less than or equal to 1, and a value of γ adopted by the first denoising process is greater than a value of γ adopted by the second denoising process.
5. The method of claim 4, wherein the first denoising process employs a larger filter radius than the second denoising process.
6. The method for noise reduction in thermal imaging according to claim 5, wherein the filter radius in the first noise reduction processing is calculated by:
7. The method of claim 1, wherein the performing face detection on the RGB image comprises:
performing multi-scale transformation on the RGB image to construct an image pyramid;
processing the image pyramid by adopting a convolutional neural network, and identifying to obtain a plurality of face windows and probability values corresponding to the face windows;
inversely transforming the face window with the probability value larger than the preset probability value to the RGB image to form a plurality of face frames;
and eliminating the crossed and repeated face frames through a non-maximum suppression algorithm to obtain the face area in the RGB image.
8. The method for noise reduction in thermal imaging according to claim 1, wherein the determining the face temperature array data in the thermal imaging temperature array data from the face region in the RGB image comprises:
acquiring coordinates of the face area in the RGB image according to the face area in the RGB image;
converting the coordinates of the face area in the RGB image to obtain the coordinates of the face temperature array data in the thermal imaging temperature array data according to the corresponding relation between the coordinates of the face area in the RGB image and the coordinates of the face temperature array data in the thermal imaging temperature array data;
and determining the face temperature array data according to the coordinates of the face temperature array data.
9. An electronic device, comprising a memory and a processor, the memory storing a computer program executable by the processor for implementing the steps of the method of noise reduction processing for thermal imaging according to any one of claims 1-8.
10. The thermal imaging noise reduction processing system is characterized by comprising a background server, an electronic terminal, a camera and a thermal imaging detector, wherein the background server is in communication connection with the electronic terminal and the camera, and the electronic terminal is in communication connection with the camera;
the camera and the thermal imaging detector are respectively used for acquiring RGB (red, green and blue) images and thermal imaging temperature array data at the same moment and uploading the RGB images and the thermal imaging temperature array data to the background server;
the background server is used for carrying out face detection on the RGB image to acquire a face area in the RGB image; determining face temperature array data in the thermal imaging temperature array data through the face region in the RGB image according to the position corresponding relation between the face region in the RGB image and the face temperature array data in the thermal imaging temperature array data; carrying out first noise reduction processing on non-human face temperature array data in the thermal imaging temperature array data to obtain thermal imaging temperature array data subjected to noise reduction processing;
and the electronic terminal is used for converting the thermal imaging temperature array data subjected to noise reduction processing into an image for displaying.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110309229.3A CN112967204A (en) | 2021-03-23 | 2021-03-23 | Noise reduction processing method and system for thermal imaging and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110309229.3A CN112967204A (en) | 2021-03-23 | 2021-03-23 | Noise reduction processing method and system for thermal imaging and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112967204A true CN112967204A (en) | 2021-06-15 |
Family
ID=76279484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110309229.3A Pending CN112967204A (en) | 2021-03-23 | 2021-03-23 | Noise reduction processing method and system for thermal imaging and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112967204A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113375811A (en) * | 2021-06-17 | 2021-09-10 | 广东天波信息技术股份有限公司 | Thermal imaging pseudo color adding method, temperature measuring device and computer storage medium |
| CN114970705A (en) * | 2022-05-20 | 2022-08-30 | 深圳市有一说一科技有限公司 | Driving state analysis method, device, equipment and medium based on multi-sensing data |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050008215A1 (en) * | 1999-12-02 | 2005-01-13 | Shepard Steven M. | System for generating thermographic images using thermographic signal reconstruction |
| KR101583773B1 (en) * | 2014-10-29 | 2016-01-20 | 동국대학교 산학협력단 | Apparatus and method for object detection |
| CN105447827A (en) * | 2015-11-18 | 2016-03-30 | 广东欧珀移动通信有限公司 | Image noise reduction method and system thereof |
| CN205212950U (en) * | 2012-12-26 | 2016-05-04 | 菲力尔系统公司 | Image device with but regionalized bolometer |
| CN108230270A (en) * | 2017-12-28 | 2018-06-29 | 努比亚技术有限公司 | A kind of noise-reduction method, terminal and computer readable storage medium |
| CN109377469A (en) * | 2018-11-07 | 2019-02-22 | 永州市诺方舟电子科技有限公司 | A kind of processing method, system and the storage medium of thermal imaging fusion visible images |
| CN111339951A (en) * | 2020-02-26 | 2020-06-26 | 北京迈格威科技有限公司 | Body temperature measuring method, device and system |
| CN111723803A (en) * | 2020-06-30 | 2020-09-29 | 广州繁星互娱信息科技有限公司 | Image processing method, device, equipment and storage medium |
| CN112132839A (en) * | 2020-08-27 | 2020-12-25 | 杭州英歌智达科技有限公司 | Multi-scale rapid face segmentation method based on deep convolution cascade network |
-
2021
- 2021-03-23 CN CN202110309229.3A patent/CN112967204A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050008215A1 (en) * | 1999-12-02 | 2005-01-13 | Shepard Steven M. | System for generating thermographic images using thermographic signal reconstruction |
| CN205212950U (en) * | 2012-12-26 | 2016-05-04 | 菲力尔系统公司 | Image device with but regionalized bolometer |
| KR101583773B1 (en) * | 2014-10-29 | 2016-01-20 | 동국대학교 산학협력단 | Apparatus and method for object detection |
| CN105447827A (en) * | 2015-11-18 | 2016-03-30 | 广东欧珀移动通信有限公司 | Image noise reduction method and system thereof |
| CN108230270A (en) * | 2017-12-28 | 2018-06-29 | 努比亚技术有限公司 | A kind of noise-reduction method, terminal and computer readable storage medium |
| CN109377469A (en) * | 2018-11-07 | 2019-02-22 | 永州市诺方舟电子科技有限公司 | A kind of processing method, system and the storage medium of thermal imaging fusion visible images |
| CN111339951A (en) * | 2020-02-26 | 2020-06-26 | 北京迈格威科技有限公司 | Body temperature measuring method, device and system |
| CN111723803A (en) * | 2020-06-30 | 2020-09-29 | 广州繁星互娱信息科技有限公司 | Image processing method, device, equipment and storage medium |
| CN112132839A (en) * | 2020-08-27 | 2020-12-25 | 杭州英歌智达科技有限公司 | Multi-scale rapid face segmentation method based on deep convolution cascade network |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113375811A (en) * | 2021-06-17 | 2021-09-10 | 广东天波信息技术股份有限公司 | Thermal imaging pseudo color adding method, temperature measuring device and computer storage medium |
| CN114970705A (en) * | 2022-05-20 | 2022-08-30 | 深圳市有一说一科技有限公司 | Driving state analysis method, device, equipment and medium based on multi-sensing data |
| CN114970705B (en) * | 2022-05-20 | 2024-05-07 | 深圳市有一说一科技有限公司 | Running state analysis method, device, equipment and medium based on multi-sensing data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110428366B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| US10719958B2 (en) | Methods and systems for fusion display of thermal infrared and visible image | |
| Hu et al. | An adaptive fusion algorithm for visible and infrared videos based on entropy and the cumulative distribution of gray levels | |
| CN110580487A (en) | Neural network training method, neural network construction method, image processing method and device | |
| WO2019133922A1 (en) | Point cloud denoising systems and methods | |
| Zhu et al. | Multiscale infrared and visible image fusion using gradient domain guided image filtering | |
| CN109919861A (en) | Infrared image enhancing method, device, computer equipment and storage medium | |
| CN112967204A (en) | Noise reduction processing method and system for thermal imaging and electronic equipment | |
| WO2021253940A1 (en) | Systems and methods for object measurement | |
| US9501816B2 (en) | Reducing the dynamic range of image data | |
| WO2022165722A1 (en) | Monocular depth estimation method, apparatus and device | |
| CN111507298A (en) | Face detection method and device, computer equipment and storage medium | |
| CN113379609B (en) | Image processing method, storage medium and terminal equipment | |
| Trongtirakul et al. | Unsupervised and optimized thermal image quality enhancement and visual surveillance applications | |
| CN117575953B (en) | Detail enhancement method for high-resolution forestry remote sensing image | |
| US11605220B2 (en) | Systems and methods for video surveillance | |
| CN114581318A (en) | Low-illumination image enhancement method and system | |
| CN110633759A (en) | Image fusion method and device and electronic equipment | |
| CN107424134B (en) | Image processing method, image processing device, computer-readable storage medium and computer equipment | |
| CN113222862A (en) | Image distortion correction method, device, electronic equipment and storage medium | |
| CN113159229A (en) | Image fusion method, electronic equipment and related product | |
| WO2023185646A1 (en) | Systems and methods for image processing | |
| Peng et al. | Hyperspectral image enhancement with vector bilateral filtering | |
| WO2020051897A1 (en) | Image fusion method and system, electronic device, and computer readable storage medium | |
| Jiang et al. | Perceptual-based fusion of ir and visual images for human detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |