CN116645310A - Method, device and storage medium for processing image - Google Patents
Method, device and storage medium for processing image Download PDFInfo
- Publication number
- CN116645310A CN116645310A CN202210137854.9A CN202210137854A CN116645310A CN 116645310 A CN116645310 A CN 116645310A CN 202210137854 A CN202210137854 A CN 202210137854A CN 116645310 A CN116645310 A CN 116645310A
- Authority
- CN
- China
- Prior art keywords
- image
- temperature
- images
- pixel point
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 124
- 238000012545 processing Methods 0.000 title claims abstract description 43
- 238000012937 correction Methods 0.000 claims abstract description 120
- 230000000694 effects Effects 0.000 claims abstract description 16
- 238000004590 computer program Methods 0.000 claims description 12
- 238000003384 imaging method Methods 0.000 abstract description 8
- 238000004891 communication Methods 0.000 abstract description 3
- 238000003331 infrared imaging Methods 0.000 description 43
- 230000008569 process Effects 0.000 description 37
- 238000012549 training Methods 0.000 description 26
- 230000003287 optical effect Effects 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000008030 elimination Effects 0.000 description 3
- 238000003379 elimination reaction Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000004297 night vision Effects 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
The application discloses a method, a device and a storage medium for processing images, and belongs to the field of communication. The method comprises the following steps: acquiring a first image and a first temperature, wherein the first image is an image acquired by an infrared detector, and the first temperature is the temperature of the infrared detector when acquiring the first image; acquiring correction parameters of the first image based on the first temperature; and eliminating non-uniformity noise contained in the first image based on the correction parameters to obtain a second image. The method and the device can better eliminate non-uniform noise in the infrared image and improve the imaging effect of the infrared image.
Description
Technical Field
The present application relates to the field of communications, and in particular, to a method, an apparatus, and a storage medium for processing an image.
Background
The infrared light is not influenced by weather and light, has good penetrability, can penetrate through large fog, dense smoke, haze and the like, and can solve the problems of night vision low illumination, long-distance small target detection and the like. Infrared light is applied to infrared cameras because of its many advantages.
However, the infrared image captured by the infrared camera often has a non-uniform noise, for example, the non-uniform noise may be "pot cover" noise, and the noise may be represented by dark surrounding and bright middle of the infrared image. In order to eliminate non-uniformity noise in the infrared image, the shot infrared image needs to be corrected at present, but the accuracy of the correction of the infrared image is low at present, that is, the non-uniformity noise in the infrared image cannot be well eliminated, so that the imaging effect of the infrared image is poor.
Disclosure of Invention
The application provides a method, a device and a storage medium for processing images, which are used for improving the imaging effect of infrared images. The technical scheme is as follows:
in a first aspect, the application provides a method of processing an image in which a first image is acquired, the first image being an image acquired by an infrared detector, and a first temperature being a temperature of the infrared detector at the time the first image was acquired. A correction parameter of the first image is acquired based on the first temperature. Based on the correction parameter, non-uniformity noise contained in the first image is eliminated to obtain a second image.
The first temperature is the temperature of the image acquired by the infrared detector, and the correction parameter of the first image is acquired based on the first temperature, and corresponds to the first temperature, so that the non-uniformity noise contained in the first image is eliminated based on the correction parameter, the accuracy of correcting the infrared image can be improved, namely the non-uniformity noise in the infrared image can be better eliminated, and the imaging effect of the infrared image is improved.
In one possible implementation, the correction parameters of the first image include correction parameters of the first pixel, the correction parameters of the first pixel being used to correct non-uniformity noise contained in the first pixel, the first image including the first pixel. And acquiring correction parameters of the first pixel point based on the first corresponding relation, the index of the first pixel point and the first temperature. The first correspondence relationship includes a correspondence relationship among a temperature, an index of the pixel point, and a correction parameter, so that the correction parameter opposite to the first temperature can be rapidly obtained through the first correspondence relationship.
In another possible implementation, the correction parameter of the first pixel point includes a gain coefficient of the first pixel point, and/or a temperature drift coefficient of the first pixel point, where the gain coefficient is used to correct the gain of the first pixel point, and the temperature drift coefficient is used to correct the effect of the first temperature on the first pixel point. Thus, the gain coefficient and the temperature drift coefficient are used for correcting the first pixel point, so that the gain of the first pixel point is improved, and the influence of the first temperature on the first pixel point is reduced.
In another possible implementation manner, M frames of third images and N frames of fourth images are acquired, where M and N are integers greater than 1, the M frames of third images are images acquired by the infrared detector at the second temperature of the calibration instrument at the third temperature, and the N frames of fourth images are images acquired by the infrared detector at the second temperature of the calibration instrument at the fourth temperature, and the third temperature is less than the fourth temperature. And acquiring a first corresponding relation based on the M frames of third images and the N frames of fourth images. Based on the M-frame third image and the N-frame fourth image, correction parameters of different pixel points at a second temperature can be obtained, and the second temperature and the correction parameters of different pixel points are stored in a first corresponding relation. Therefore, when the non-uniformity noise in the infrared image acquired by the infrared detector is corrected, the correction parameters of the infrared image can be quickly acquired based on the first corresponding relation, and the efficiency of correcting the infrared image is improved.
In another possible implementation, the correction parameter of the second pixel point is obtained based on M pixel values of the second pixel point and N pixel values of the second pixel point, where the M pixel values include pixel values of the second pixel point in the third image of each frame, and the N pixel values include pixel values of the second pixel point in the fourth image of each frame. And correspondingly storing the second temperature, the index of the second pixel point and the correction parameter of the second pixel point in the first corresponding relation. Thus, based on the M pixel values and the N pixel values, the correction parameters of the second pixel point at the second temperature can be obtained, and the correction parameters of the second temperature and the second pixel point are stored in the first corresponding relation. Therefore, when the non-uniformity noise in the infrared image acquired by the infrared detector is corrected, the correction parameters of the pixel points in the infrared image can be rapidly acquired based on the first corresponding relation, and the efficiency of correcting the infrared image is improved.
In another possible implementation, a first pixel value average is obtained based on the M pixel values, and a second pixel value average is obtained based on the N pixel values, wherein a temperature drift coefficient of the second pixel point is equal to the first pixel value average. The first total pixel value average value is obtained based on the pixel value of each pixel point included in the third image of each frame, and the second total pixel value average value is obtained based on the pixel value of each pixel point included in the fourth image of each frame. And acquiring a gain coefficient of the second pixel point based on the first pixel value average value, the second pixel value average value, the first total pixel value average value and the second total pixel value average value. Thus, based on the M pixel values and the N pixel values, the correction parameters of the second pixel point at the second temperature can be obtained
In another possible implementation manner, the first temperature is acquired by a temperature sensor, and the first temperature is the temperature of the infrared detector, and the temperature of the infrared detector can be accurately acquired by the temperature sensor.
In another possible implementation, the temperature sensor is integrated with the infrared detector, so that the accuracy of acquiring the first temperature may be improved.
In another possible implementation, the removing of the non-uniform noise contained in the second image results in the fifth image based on a priori knowledge information reflecting the difference between the noisy image and the noiseless image. And because the non-uniformity noise contained in the second image is eliminated for the second time based on the priori knowledge information, a fifth image is obtained, so that the accuracy of correcting the image is improved, the accuracy of eliminating the non-uniformity noise is improved, and the fifth image with higher definition than the second image is obtained.
In another possible implementation, a noise image is constructed based on the a priori knowledge information and the second image. Based on the noise image, non-uniformity noise contained in the second image is eliminated to obtain a fifth image. Thus, the non-uniform noise contained in the second image is eliminated based on the prior knowledge information.
In another possible implementation, a first data set and a second data set are acquired, the first data set including at least one pair of visible light images, any pair of the visible light images in the first data set including noisy and noiseless visible light images captured of a same object, the second data set including at least one pair of infrared light images, and any pair of the infrared light images in the second data set including noisy and noiseless infrared light images captured of the same object. The prior knowledge information is obtained based on the first data set and the second data set. Because any pair of images in the first data set comprises a noisy visible light image and a noiseless visible light image which are shot on the same object, and any pair of infrared light images in the second data set comprises a noisy infrared light image and a noiseless infrared light image which are shot on the same object, prior knowledge information can be accurately acquired based on the first data set and the second data set, namely the acquired prior knowledge information can accurately reflect the difference between the noisy image and the noiseless image.
In another possible implementation, the model is acquired based on a priori knowledge, and the a priori knowledge information is acquired for the first data set and the second data set. The efficiency and the accuracy of acquiring the priori knowledge information can be improved by acquiring the model through the priori knowledge.
In another possible implementation, the intelligent algorithm is trained based on at least one training sample to obtain the a priori knowledge acquisition model. Any one of the at least one training sample comprises a third data set, a fourth data set and priori knowledge information corresponding to the third data set and the fourth data set, the third data set comprises at least one pair of visible light images, any pair of visible light images in the third data set comprises noisy visible light images and noiseless visible light images shot on the same object, the fourth data set comprises at least one pair of infrared light images, and any pair of infrared light images in the fourth data set comprises noisy infrared light images and noiseless infrared light images shot on the same object. Therefore, the prior knowledge acquisition model can be trained through at least one training sample, and the efficiency and the accuracy for acquiring the prior knowledge information can be improved through the prior knowledge acquisition model.
In a second aspect, the present application provides an apparatus for processing an image for performing the method of the first aspect or any one of the possible implementations of the first aspect. In particular, the apparatus comprises means for performing the method of the first aspect or any one of the possible implementations of the first aspect.
In a third aspect, the present application provides an apparatus for processing an image, the apparatus comprising a processor and a memory, the memory for storing a computer program, the processor for executing the computer program in the memory, such that the apparatus performs the method of the first aspect or any possible implementation of the first aspect.
In a fourth aspect, the present application provides a computer program product comprising a computer program and loaded by a computer to implement the method of the first aspect or any of the possible implementations of the first aspect.
In a fifth aspect, the present application provides a computer readable storage medium storing a computer program to be loaded by a computer to perform the method of the first aspect or any possible implementation of the first aspect.
In a sixth aspect, the present application provides a chip comprising a memory for storing computer instructions and a processor for calling and executing the computer instructions from the memory to perform the method of the first aspect or any possible implementation of the first aspect.
Drawings
Fig. 1 is a schematic structural view of an infrared imaging apparatus according to an embodiment of the present application;
fig. 2 is a schematic structural view of another infrared imaging apparatus provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a network architecture according to an embodiment of the present application;
FIG. 4 is a flow chart of a method of processing an image according to an embodiment of the present application;
FIG. 5 is a flowchart of a method for eliminating non-uniformity noise in an image according to an embodiment of the present application;
FIG. 6 is a flowchart of a method for obtaining a first correspondence provided in an embodiment of the present application;
FIG. 7 is a flowchart of a method for obtaining a priori knowledge information, in accordance with an embodiment of the present application;
fig. 8 is a schematic view of an apparatus for processing an image according to an embodiment of the present application;
fig. 9 is a schematic diagram of an apparatus for processing an image according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
The current method for eliminating the image needs to set a baffle in the infrared camera equipment, and the baffle is controlled to be positioned between the infrared detector and the lens every a period of time (for example, every ten minutes), so that the baffle is positioned on the optical transmission path between the infrared detector and the lens. The shutter needs to stay on the optical transmission path for a period of several seconds (a period of 2 to 5 seconds) during which the correction parameter is acquired by the shutter. The baffle is then removed from the optical transmission path, and then non-uniformity noise contained in each frame of image acquired by the infrared detector over a period of time is eliminated based on the correction parameters.
Each frame of image acquired by the infrared detector comprises pot cover noise and/or stripe non-uniformity noise and the like. The noise of the pot cover is shown as dark surrounding and bright middle of the infrared image. Due to the inconsistent pixel response caused by the characteristics of the infrared detector material, the output image has fixed stripe non-uniformity noise. The non-uniformity noise includes pot cover noise and/or streak non-uniformity noise, etc.
The existing method increases the volume, weight, power consumption and manufacturing cost of the infrared camera equipment due to the existence of the baffle. The process of obtaining the correction parameters is cumbersome and the operability is poor. When the correction parameters are acquired each time, the baffle needs to stay on the optical transmission path for a period of several seconds, so that the infrared imaging device cannot shoot the object, namely cannot image all the time. In order to solve these problems, the present application is solved by any one of the following embodiments.
Referring to fig. 1, an embodiment of the present application provides an infrared imaging apparatus 100, the infrared imaging apparatus 100 including: the processor 101, the infrared detector 102 and the lens 103, the processor 101 is connected with the infrared detector 102.
Infrared light radiated from the object passes through the lens 103 and then irradiates the infrared detector 102. The processor 101 controls the infrared detector 102 to perform infrared thermal image scanning, the infrared detector 102 scans to obtain infrared radiation energy, the infrared radiation energy obtained by scanning is converted into an electric signal, the electric signal is amplified, and the amplified electric signal is converted into a digital signal to obtain a first image. This process of acquiring the first image is a process in which infrared detector 102 images the infrared light.
In some embodiments, the first image is an infrared image. Infrared detector 102, also known as an infrared sensor, is used to image infrared light.
Infrared detector 102 may be a long wave infrared detector, a medium wave infrared detector, or a short wave infrared detector. The wavelength range of the long-wave infrared light is a range of 7um or more and 14um or less, the wavelength range of the medium-wave infrared light is a range of 3um or more and 3um or less, and the wavelength range of the short-wave infrared light is a range of 1.1um or more and 2.5um or less. Optionally, the primary operating wavelength range of the long wave infrared light detector is a range of greater than or equal to 8um and less than or equal to 14 um.
In some embodiments, the processor 101 is a micro control unit (microcontroller unit, MCU), a single chip or field programmable gate array (field programmable gate array, FPGA), or the like.
In some embodiments, processor 101 acquires a first image acquired by infrared detector 102, performs image signal processing (image signal processing, ISP) processing and/or noise cancellation processing, etc., on first image 1.
In some embodiments, referring to fig. 2, the infrared imaging apparatus 100 further includes an ISP chip 104, and the ISP chip 104 is connected to the processor 101 and the infrared detector 102, respectively. After the first image is acquired by the infrared detector 102, the first image is input to the ISP chip 104, the ISP chip 104 performs ISP processing on the first image, and the processed first image is input to the processor 101. The processor 101 receives the first image, performs noise cancellation processing on the first image to obtain a second image, and the like.
In some embodiments, after ISP processing is performed on the first image, ISP chip 104 may also perform noise cancellation processing on the first image after ISP processing to obtain a second image, and input the second image to processor 101.
In some embodiments, referring to fig. 2, the infrared camera apparatus 100 further includes a display unit 105, the display unit 105 being connected to the processor 101, the processor 101 being configured to display the second image on the display unit 105.
In some embodiments, the display component 105 is a display screen or the like.
Wherein the temperature of the infrared detector 102 may increase while the plurality of images are continuously acquired by the infrared detector 102. The non-uniformity noise contained in the images acquired by the infrared detector 102 at different temperatures is different.
In some embodiments, the non-uniformity noise may comprise pot cover noise. The pot cover noise can cause dark surrounding and bright middle of the image, so that the brightness of the image is uneven, and the definition of the image display is affected. And/or, the non-uniformity noise may comprise streak non-uniformity noise. The streak non-uniformity noise can cause streaks in the image, thereby affecting the sharpness of the image display.
In order to eliminate non-uniformity noise in the image, referring to fig. 2, a temperature sensor 106 is disposed on the infrared detector 102, that is, the infrared detector 102 and the temperature sensor 106 are integrated together, the temperature sensor 106 is connected to the processor 101, and the temperature sensor 106 is used for measuring the temperature of the infrared detector 102 in real time.
Thus, when the infrared detector 102 acquires the first image, the temperature sensor 106 acquires the first temperature when the infrared detector 102 acquires the first image, acquires the correction parameter of the first image based on the first temperature, and eliminates the non-uniformity noise in the first image based on the correction parameter of the first image. At this time, the first elimination of the non-uniformity noise contained in the image acquired by the infrared detector 102 is realized.
In some embodiments, the correction parameters of the first image include correction parameters for each pixel in the first image. For convenience of explanation, for any one pixel point in the first image, the pixel point is referred to as a first pixel point, the first pixel point contains non-uniformity noise, and a correction parameter of the first pixel point is used for correcting the non-uniformity noise contained in the first pixel point. Optionally, the operation of acquiring the correction parameters of the first image is:
and acquiring correction parameters of each pixel point in the first image based on the first corresponding relation, the first temperature and the index of each pixel point in the first image, wherein the first corresponding relation comprises the corresponding relation among the temperature, the index of each pixel point and the correction parameters.
In some embodiments, the index of the first pixel includes a row number and a column number of the first pixel in the first image. For example, the first pixel is the pixel of the ith row and jth column in the first image, so the index of the first pixel includes a row number "i" and a column number "j". The index of the first pixel point may be denoted as (i, j).
In some embodiments, the processor 101 or ISP chip 104 in the infrared imaging apparatus 100 acquires correction parameters of the first image based on the first temperature, and removes non-uniformity noise in the first image based on the correction parameters of the first image.
In some embodiments, referring to fig. 2, the infrared imaging apparatus 100 further includes a memory 107, the processor 101 is connected to the memory 107, and the memory 107 includes a first correspondence.
When the processor 101 eliminates the non-uniformity noise in the first image, the processor 101 acquires the first temperature at which the infrared detector 102 acquires the first image through the temperature sensor 106, acquires the correction parameter of the first image from the memory 107 based on the first temperature, and then eliminates the non-uniformity noise in the first image using the correction parameter of the first image.
When the ISP chip 104 eliminates the non-uniformity noise in the first image, the processor 101 acquires the first temperature at which the infrared detector 102 acquires the first image through the temperature sensor 106, acquires the correction parameters of the first image from the memory 107 based on the first temperature, and then inputs the correction parameters of the first image to the ISP chip 104. ISP chip 104 receives the correction parameters and uses the correction parameters to eliminate non-uniformity noise in the image.
In some embodiments, M frames of third images and N frames of fourth images are acquired, where M and N are integers greater than 1, the M frames of third images are images acquired by the infrared detector 102 at the second temperature of the calibration instrument at the third temperature, and the N frames of fourth images are images acquired by the infrared detector 102 at the second temperature of the calibration instrument at the fourth temperature, and the third temperature is less than the fourth temperature; based on the M-frame third image and the N-frame fourth image, a first correspondence is acquired, and then stored in the memory 107.
In some embodiments, to improve the sharpness of the image, processor 101 or ISP chip 104 may also perform a second cancellation of non-uniformity noise contained in the image acquired by infrared detector 102, resulting in a fifth image having a higher sharpness than the second image. In implementation, the processor 101 or ISP chip 104 eliminates non-uniform noise contained in the second image based on a priori knowledge information reflecting the difference between the noisy image and the noiseless image to obtain a fifth image.
In some embodiments, memory 107 includes the a priori knowledge information that processor 101 or ISP chip 104 obtains from memory 107 when performing a second cancellation of non-uniform noise contained in the image acquired by infrared detector 102.
Referring to fig. 3, an embodiment of the present application provides a network architecture 300, the network architecture 300 comprising: the first device 301 and the infrared imaging device 100 shown in fig. 1 or 2, the first device 301 communicates with the infrared imaging device 100.
In some embodiments, for a first image acquired by infrared detector 102 in infrared imaging device 100, infrared imaging device 100 may not cancel non-uniformity noise in the first image, but rather non-uniformity noise in the first image is cancelled by first device 301. When the method is realized, the following steps are:
the infrared imaging apparatus 100 acquires a first image acquired by the infrared detector 102 in the infrared imaging apparatus 100 and a first temperature at the time of acquiring the first image, and transmits the first image and the first temperature to the first apparatus 301. The first device 301 receives the first temperature and the first image, acquires a correction parameter of the first image based on the first temperature, and eliminates non-uniformity noise in the first image based on the correction parameter to obtain a second image.
In some embodiments, the first device 301 may also perform second cancellation of non-uniformity noise in the image acquired by the infrared imaging device 100, resulting in a fifth image having higher sharpness than the second image. In implementation, the first device 301 eliminates non-uniform noise contained in the second image based on the a priori knowledge information, resulting in a fifth image.
In some embodiments, the first device 301 may include the first correspondence, and the first device 301 obtains the correction parameters of the first image, that is, obtains the correction parameters of each pixel in the first image, based on the first correspondence, the first temperature, and the index of each pixel in the first image. And/or the first device 301 may include the a priori knowledge information.
In some embodiments, for a first image acquired by infrared detector 102 in infrared imaging device 100, infrared imaging device 100 may cancel non-uniformity noise in the first image to obtain a second image, or cancel non-uniformity noise in the second image to obtain a fifth image, and then send the second image or the fifth image to first device 301.
In some embodiments, the first device 301 performs a specified operation on the second image or the fifth image after obtaining the second image or the fifth image. For example, the network architecture 300 is applied to the security field, and the first device 301 displays the second image or the fifth image, so that the first device 301 obtains the second image or the fifth image through the infrared camera device 100, and displays the second image or the fifth image to the user, thereby realizing real-time monitoring.
For another example, the network architecture 300 is applied to the intelligent driving field, where the first device 301 and the infrared camera device 100 are both installed on a car, and the first device 301 may be a car machine installed on the car or a controller of the car. Thus, the first apparatus 301 acquires the second image or the fifth image through the infrared imaging apparatus 100, and performs automatic driving based on the second image or the fifth image.
The network architecture 300 may also have application in other areas, not specifically recited herein.
For a detailed implementation procedure of the infrared imaging apparatus 100 or the first apparatus 301 to eliminate the non-uniformity noise in the first image, see by any one of the embodiments as follows.
Referring to fig. 4, an embodiment of the present application provides a method 400 of processing an image, the method 400 being applied to the infrared camera apparatus 100 of the embodiment shown in fig. 1 or fig. 2, or the method 400 being applied to the network architecture 300 shown in fig. 3. The execution subject of the method 400 is a processor or ISP chip in the infrared camera device 100, or the execution subject of the method 400 is the first device 301 in the network architecture 300. The method 400 includes the following steps.
Step 401: and acquiring a first image and a first temperature, wherein the first image is an image acquired by the infrared detector, and the first temperature is the temperature of the infrared detector when the first image is acquired.
The infrared detector is provided with a temperature sensor, and when a first image acquired by the infrared detector is acquired, the current temperature of the infrared detector is measured by the temperature sensor, and the current temperature is the first temperature.
When the execution subject of the method 400 is a processor or an ISP chip in the infrared imaging apparatus 100 and the infrared detector acquires the first image, the first image is input to the processor or the ISP chip. The processor or ISP chip receives the first image and acquires the current temperature measured by the temperature sensor as the first temperature.
In the implementation of the method 400, where the subject is a first device, after a processor of the infrared imaging device acquires a first temperature and a first image, the infrared imaging device sends the first temperature and the first image to the first device. The first device receives a first temperature and a first image.
Step 402: a correction parameter of the first image is acquired based on the first temperature.
In some embodiments, the correction parameters of the first image include correction parameters for each pixel in the first image. For any pixel point in the first image, the pixel point is called a first pixel point, and the correction parameter of the first pixel point comprises a gain coefficient of the first pixel point and/or a temperature drift coefficient of the first pixel point. The gain coefficient of the first pixel point is used for correcting the gain of the first pixel point, and the temperature drift coefficient of the first pixel point is used for correcting the influence of the first temperature on the first pixel point.
In step 402, a correction parameter for each pixel in the first image is obtained based on the first temperature, the index of each pixel in the first image, and the first correspondence.
Taking a first pixel point in a first image as an example, based on a first temperature, an index of the first pixel point and a first corresponding relation, the operation of obtaining a correction parameter of the first pixel point is as follows:
at least one temperature corresponding to the index of the first pixel point is obtained from the first corresponding relation, a second temperature is selected from the at least one temperature based on the first temperature, the second temperature is the temperature with the minimum difference between the at least one temperature and the first temperature, and the corresponding correction parameter is searched from the first corresponding relation to serve as the correction parameter of the first pixel point based on the second temperature and the index of the first pixel point.
In some embodiments, the temperature at which the difference between the at least one temperature and the first temperature is smallest may include two temperatures, and one temperature is randomly selected from the two temperatures as the second temperature. Or taking the two temperatures as the second temperatures, namely two second temperatures, searching two corresponding correction parameters from the first corresponding relation based on the indexes of the two second temperatures and the first pixel point, calculating the average value of the two correction parameters, and taking the average value as the correction parameter of the first pixel point.
The two correction parameters comprise two gain coefficients and/or two temperature drift coefficients, and the correction parameter of the first pixel point comprises the average value of the two gain coefficients and/or the average value of the two temperature drift coefficients.
For example, the first apparatus or the infrared imaging apparatus includes a first correspondence as shown in table 1 below, a first record in the first correspondence includes temperature 1, an index (1, 1) of a pixel, which indicates that the pixel is a pixel of a 1 st row and a 1 st column in an image, and a correction parameter (including gain coefficient K11 and temperature drift coefficient B11) which is a correction parameter of the pixel when the temperature of the infrared detector is temperature 1.
The first record in the first correspondence relationship includes temperature 2, index (1, 1) of the pixel point, and correction parameters (including gain coefficient K12 and temperature drift coefficient B12) that are correction parameters of the pixel point when the temperature of the infrared detector is temperature 2. The meanings of the other records in the first correspondence shown in table 1 are not listed.
TABLE 1
Assuming that the first temperature is 3.6 degrees, the first pixel is the first pixel in the first row and the first column in the first image, that is, the first pixel is the first pixel in the first image, and the index of the first pixel is (1, 1). At least one temperature corresponding to the index (1, 1) of the first pixel point is obtained from the first correspondence as shown in table 1, the at least one temperature including 1 degree, 2 degrees, 3 degrees, 4 degrees, … …. The temperature 4 having the smallest difference from the first temperature "3.6" is obtained from the at least one temperature as the second temperature. Based on the second temperature "4" and the index (1, 1) of the first pixel, corresponding correction parameters including the gain coefficient K14 and the temperature drift coefficient B14 are obtained from the first correspondence as shown in table 1. That is, the correction parameters of the first pixel point include a gain coefficient K14 and a temperature drift coefficient B14.
And for the second pixel point, the third pixel point and the … … in the first image, the correction parameters of the second pixel point, the correction parameters of the third pixel point and the correction parameters of … … are obtained in the same manner so as to obtain the correction parameters of each pixel point in the first image.
Step 403: and eliminating non-uniformity noise contained in the first image based on the correction parameters of the first image to obtain a second image.
In step 403, based on the correction parameter of the first pixel, the non-uniformity noise included in the first pixel is corrected according to the following first formula.
The first formula is: y=k (X-B);
in the first formula, Y is the pixel value of the first pixel after correction, X is the pixel value of the first pixel before correction, K is the gain coefficient of the first pixel, and B is the temperature drift coefficient of the first pixel.
And correcting the non-uniformity noise contained in each other pixel point in the first image according to the first formula, so as to eliminate the non-uniformity noise contained in the first image and obtain a second image.
In the embodiment of the application, the first temperature of the first image shot by the infrared detector is acquired, the correction parameter of the first image is acquired based on the first temperature, the correction parameter corresponds to the first temperature, the non-uniformity noise contained in the first image is eliminated based on the correction parameter, the accuracy of the corrected image can be improved, the effect of eliminating the non-uniformity noise is improved, and the non-uniformity noise in the infrared image can be better eliminated, so that the imaging effect of the infrared image is improved. The correction parameters of the pixels in the first image are acquired based on the first temperature, the first corresponding relation and the index of the pixels in the first image, so that a baffle is not needed, the volume, the weight, the power consumption and the manufacturing cost of the infrared camera can be reduced, the process of acquiring the correction parameters is simplified, and all-weather imaging can be realized.
Wherein, the non-uniformity noise contained in the first image is eliminated for the first time through the steps 401-403, so as to obtain a second image. The second image may further include non-uniformity noise, and in order to improve the sharpness of the second image, referring to fig. 5, the non-uniformity noise included in the second image is eliminated for the second time, that is, based on the a priori knowledge information, to eliminate the non-uniformity noise included in the second image, so as to obtain a fifth image. The second elimination includes the following steps.
Step 501: a noise image is constructed based on the prior knowledge information and the second image.
The a priori knowledge information is used to reflect the differences between the noisy and the noiseless images. The prior knowledge information includes prior knowledge information of a plurality of pixel points. For any one pixel point, for convenience of explanation, the pixel point will be referred to as a first pixel point, and a priori knowledge information of the first pixel point is used to reflect a noise difference between the first pixel point in the noise image and the first pixel point in the noise-free image.
In step 501, noise of a first pixel is obtained based on prior knowledge information of the first pixel and a pixel value of the first pixel in a second image. And acquiring the noise of each other pixel point in the second image in the same way, and obtaining a noise image of the second image based on the noise of each pixel point in the second image.
Step 502: based on the noise image, non-uniformity noise contained in the second image is eliminated to obtain a fifth image.
In step 502, the second image is subtracted from the noise image to obtain a fifth image with non-uniformity noise removed.
In the embodiment of the application, the non-uniformity noise contained in the second image is eliminated for the second time based on the priori knowledge information to obtain the fifth image, so that the accuracy of correcting the image is improved, the effect of eliminating the non-uniformity noise is improved, the fifth image with higher definition than the second image is obtained, and the imaging effect of the infrared image is further improved.
For the first correspondence, the embodiment of the present application provides a method 600 for acquiring the first correspondence, where the method 600 is applied to the infrared camera device 100 in the embodiment shown in fig. 1 or fig. 2, or the method 600 is applied to the network architecture 300 shown in fig. 3. The execution subject of the method 600 is a processor or ISP chip in the infrared camera device 100, or the execution subject of the method 600 is the first device 301 in the network architecture 300. The method 600 includes the following steps.
Step 601: and acquiring M frames of third images, wherein M is an integer greater than 1, and the M frames of third images are images obtained by acquiring a calibration instrument at a third temperature when the infrared detector is at a second temperature.
In step 601, the temperature of the calibration apparatus is controlled to be kept at a third temperature, and then the calibration apparatus is continuously acquired using an infrared detector. And acquiring a frame of image from the infrared detector, and acquiring the image and the current temperature of the infrared detector measured by the temperature sensor. The correspondence between the frame image and the temperature is saved in a second correspondence.
The third temperature may be 20 degrees, 30 degrees, 40 degrees, or the like.
The temperature of the infrared detector may change continuously during the continuous acquisition of the calibration instrument by the infrared detector. For example, referring to table 2 below, assuming that the temperature of the current infrared detector is 1 degree, the infrared detector continuously acquires a plurality of frames of images including an image A1, an image A2, an image A3 and an image A4 when the infrared detector is at 1 degree, and the 1 degree and the plurality of frames of images are correspondingly stored in a second correspondence relationship as shown in table 2. The infrared detector continuously collects images, when the temperature of the infrared detector is increased to 1.3 ℃, the infrared detector continuously collects multiple frames of images including an image A5, an image A6, an image A7, an image A8 and an image A9, and the 1.3-degree images and the multiple frames of images are correspondingly stored in a second corresponding relation shown in the table 2. When the temperature of the infrared detector is not continuously increased, the infrared detector can be stopped from continuously collecting images of the calibration instrument at the third temperature.
TABLE 2
| Sequence number | Temperature (temperature) | Image processing apparatus |
| 1 | 1 degree | Image A1, image A2, image A3, and image A4 |
| 2 | 1.3 degree | Image A5, image A6, image A7, image A8, image A9 |
| 3 | 1.5 degree | Image a10, image a11, image a12 |
| 4 | 1.8 degree | Image a13, image a14, image a15, image a16, image a17 |
| 5 | 2 degrees | Image A18, image A19, image A20 |
| …… | …… | …… |
In step 601, one temperature is selected from the temperatures included in the second correspondence as the second temperature, and an M-frame image corresponding to the second temperature is acquired from the second correspondence, where the M-frame image is an M-frame third image.
For example, 1 degree is selected as the second temperature from among 1 degree, 1.3 degrees, 1.5 degrees, 1.8 degrees, 2 degrees, … … included in the second correspondence relation as shown in table 2. A4-frame third image corresponding to 1 degree, that is, m=4, is acquired from the second correspondence relationship shown in table 2, and the 4-frame third image is the image A1, the image A2, the image A3, and the image A4.
Step 602: and acquiring N frames of fourth images, wherein N is an integer larger than 1, and the N frames of fourth images are images obtained by acquiring a calibration instrument at the fourth temperature when the infrared detector is at the second temperature.
In step 602, the temperature of the calibration apparatus is controlled to be maintained at a fourth temperature, and then the calibration apparatus is continuously acquired using an infrared detector. And acquiring a frame of image from the infrared detector, and acquiring the image and the current temperature of the infrared detector measured by the temperature sensor. The correspondence between the frame image and the temperature is saved in a third correspondence.
The third temperature is less than the fourth temperature, which may be 60 degrees, 70 degrees, 80 degrees, or the like.
The temperature of the infrared detector may change continuously during the continuous acquisition of the calibration instrument by the infrared detector. For example, referring to table 3 below, assuming that the temperature of the current infrared detector is 1 degree, the infrared detector continuously acquires a plurality of frames of images including an image B1, an image B2, an image B3, an image B4, and an image B5 while the infrared detector is at 1 degree, and the 1 degree and the plurality of frames of images are correspondingly stored in a third correspondence relationship as shown in table 3. The infrared detector continuously collects images, when the temperature of the infrared detector is increased to 1.3 ℃, the infrared detector continuously collects multiple frames of images including an image B6, an image B7, an image B8 and an image B9, and the 1.3-degree images and the multiple frames of images are correspondingly stored in a third corresponding relation shown in the table 3. When the temperature of the infrared detector is not continuously increased, the infrared detector can be stopped from continuously collecting images of the calibration instrument at the fourth temperature.
TABLE 3 Table 3
| Sequence number | Temperature (temperature) | Image processing apparatus |
| 1 | 1 degree | Image B1, image B2, image B3, image B4, image B5 |
| 2 | 1.3 degree | Image B6, image B7, image B8, image B9 |
| 3 | 1.5 degree | Image B10, image B11, image B12 |
| 4 | 1.8 degree | Image B13, image B14, image B15, image B16, image B17 |
| 5 | 2 degrees | Image B18, B19, B20 |
| …… | …… | …… |
In step 602, N frame images corresponding to the second temperature are obtained from the fourth correspondence, where the N frame images are N frame fourth images.
For example, 5 fourth images corresponding to 1 degree, that is, n=5, are acquired from the third correspondence relationship shown in table 3, and the 5 fourth images are the image B1, the image B2, the image B3, and the image B4.
The step 601 and the step 602 are not consecutive, and the step 601 may be performed first and then the step 602 may be performed, or the step 602 may be performed first and then the step 601 may be performed.
Next, the first correspondence is acquired based on the M-frame third image and the N-frame fourth image, and the detailed acquisition process includes the following steps 603 and 604.
The following description is needed: the infrared image capturing apparatus may store the second correspondence and the third correspondence, acquire M-frame third images corresponding to the second temperature from the second correspondence, acquire N-frame fourth images corresponding to the second temperature from the third correspondence, and then perform the process of acquiring the first correspondence as follows. Or alternatively, the process may be performed,
after the infrared camera equipment acquires the M-frame third image and the N-frame fourth image corresponding to the second temperature, the M-frame fourth image and the N-frame fourth image corresponding to the second temperature are sent to the first equipment. The first device then performs a process of acquiring the first correspondence as follows. Or alternatively, the process may be performed,
The infrared camera equipment acquires the second corresponding relation and the third corresponding relation and sends the second corresponding relation and the third corresponding relation to the first equipment. The first device receives the second corresponding relation and the third corresponding relation, acquires an M-frame third image corresponding to the second temperature from the second corresponding relation and acquires an N-frame fourth image corresponding to the second temperature from the third corresponding relation, and then performs the following process of acquiring the first corresponding relation. Or alternatively, the process may be performed,
the infrared camera equipment sends at least one frame of third image and at least one frame of fourth image corresponding to a certain temperature to the first equipment after acquiring the at least one frame of third image and the at least one frame of fourth image corresponding to the certain temperature. And then the first equipment receives at least one frame of third image and at least one frame of fourth image corresponding to the temperature, correspondingly stores the temperature and the at least one frame of third image in a second corresponding relation, and correspondingly stores the temperature and the at least one frame of fourth image in a third corresponding relation. Then, the first device acquires M-frame third images corresponding to the second temperature from the second correspondence and N-frame fourth images corresponding to the second temperature from the third correspondence, and performs the process of acquiring the first correspondence as follows.
Step 603: and acquiring correction parameters corresponding to the second pixel points based on M pixel values of the second pixel points and N pixel values of the second pixel points, wherein the M pixel values comprise pixel values corresponding to the second pixel points in the third image of each frame, and the N pixel values comprise pixel values corresponding to the second pixel points in the fourth image of each frame.
Assuming that the index of the second pixel point is (i, j), the pixel points of the ith row and the jth column in each frame of the third image are respectively the second pixel points of each frame of the third image, and obtaining the pixel values of the pixel points of the ith row and the jth column in each frame of the third image to obtain M pixel values of the second pixel points. Similarly, the pixel points in the ith row and the jth column in each frame of the fourth image are respectively the second pixel points in each frame of the fourth image, and the pixel values of the pixel points in the ith row and the jth column in each frame of the fourth image are obtained to obtain N pixel values of the second pixel points.
For example, for the above-described 4-frame third image (images A1, A2, A3, A4) and the above-described 5-frame fourth image (images B1, B2, B3, B4, B5), the index of the second pixel point is (i, j), and the M pixel values of the second pixel point include the pixel value of the pixel point of the ith row and the jth column in the image A1, the pixel value of the pixel point of the ith row and the jth column in the image A2, the pixel value of the pixel point of the ith row and the jth column in the image A3, and the pixel value of the pixel point of the ith row and the jth column in the image A4. The N pixel values of the second pixel point include a pixel value of a pixel point of an ith row and a jth column in the image B1, a pixel value of a pixel point of an ith row and a jth column in the image B2, a pixel value of a pixel point of an ith row and a jth column in the image B3, a pixel value of a pixel point of an ith row and a jth column in the image B4, and a pixel value of a pixel point of an ith row and a jth column in the image B5.
In step 603, the correction parameters of the second pixel point may be acquired through operations 6031 to 6033, respectively.
6031: and acquiring a first pixel value average value based on the M pixel values, and acquiring a second pixel value average value based on the N pixel values, wherein the temperature drift coefficient corresponding to the second pixel point is equal to the first pixel value average value.
6032: the first total pixel value average value is obtained based on the pixel value of each pixel point included in the third image of each frame, and the second total pixel value average value is obtained based on the pixel value of each pixel point included in the fourth image of each frame.
6033: and acquiring a gain coefficient of the second pixel point based on the first pixel value average value, the second pixel value average value, the first total pixel value average value and the second total pixel value average value.
In 6033, the gain factor of the second pixel point is obtained according to the following second formula based on the first pixel value average, the second pixel value average, the first total pixel value average, and the second total pixel value average.
The second formula is:
wherein, in the second formula, K i,j Is the gain factor of the second pixel point,for the first pixel value mean, +.>Is the second pixel value mean, +.>For the first total pixel value mean, +. >Is the second total pixel value average.
Step 604: and correspondingly storing the second temperature, the index of the second pixel point and the correction parameter of the second pixel point in the first corresponding relation.
And repeating the processes of the steps 603-604 for other temperatures and other pixel points, and correspondingly storing the other temperatures, indexes of other pixel points and correction parameters of other pixel points in a first corresponding relation.
And for the first corresponding relation, the infrared detector uses the first corresponding relation to eliminate non-uniformity noise contained in the image acquired by the infrared detector. The first correspondence relationship may also be used by other infrared detectors to eliminate non-uniformity noise contained in images acquired by the other infrared detectors.
In some embodiments, the infrared imaging device obtains the first correspondence, stores it locally or in a separate storage device, and then uses the first correspondence to perform the method 400 shown in fig. 4. Or alternatively, the process may be performed,
after the infrared camera device obtains the first correspondence, the first correspondence is sent to the first device, and then the first device executes the method 400 shown in fig. 4 described above using the first correspondence. Or alternatively, the process may be performed,
after the first device obtains the first correspondence, the first device stores the first correspondence locally or in a separate storage device, and then uses the first correspondence to execute the method 400 shown in fig. 4. Or alternatively, the process may be performed,
After the first device obtains the first correspondence, the first correspondence is saved in the infrared imaging device, and then the infrared imaging device performs the method 400 shown in fig. 4 described above using the first correspondence.
In the embodiment of the application, M frames of third images and N frames of fourth images corresponding to the second temperature are acquired; and acquiring a first corresponding relation based on the M frames of third images and the N frames of fourth images. Because the first corresponding relation comprises correction parameters corresponding to different temperatures, non-uniformity noise contained in an image shot by the infrared detector is eliminated based on the first corresponding relation, the accuracy of the corrected image is improved, and the effect of eliminating the non-uniformity noise is improved. Because the first corresponding relation comprises correction parameters corresponding to different temperatures, when non-uniformity noise contained in an image shot by the infrared detector is eliminated, the correction parameters can be obtained based on the current temperature of the infrared detector and the first corresponding relation, the infrared detector is not required to stop acquiring the image, and all-weather shooting of the infrared detector can be realized.
For the prior knowledge information described above, the embodiment of the present application provides a method 700 for acquiring the prior knowledge information, where the method 700 is applied to the infrared camera apparatus 100 of the embodiment shown in fig. 1 or fig. 2, or the method 700 is applied to the network architecture 300 shown in fig. 3. The execution subject of the method 700 is a processor in the infrared camera device 100, or the execution subject of the method 700 is the first device 301 in the network architecture 300. The method 700 includes the following steps.
Step 701: a first data set is acquired, the first data set comprises at least one pair of visible light images, and any pair of visible light images in the first data set comprises a noisy visible light image and a noiseless visible light image which are shot on the same object.
In step 701, the same object is photographed using a visible-light noisy camera and a visible-light noiseless camera to obtain a pair of visible-light images including a noisy visible-light image photographed by the visible-light noisy camera and a noiseless visible-light image photographed by the visible-light noiseless camera. Wherein, use visible light noiseless camera and visible light noiseless camera to shoot different objects, can be many to the visible light image.
The quality of the image shot by the visible light noisy camera is low, and the noisy visible light image shot by the visible light noisy camera contains noise. The image quality photographed by the visible-light noiseless camera is high, and normally the noiseless visible-light image photographed by the visible-light noiseless camera does not contain noise.
In some embodiments, the infrared camera device or the first device may be connected to a visible light noisy camera and a visible light noiseless camera, respectively. When the visible light noisy camera and the visible light noiseless camera respectively shoot the same object, the infrared camera device or the first device receives the visible light noisy image sent by the visible light noisy camera and receives the visible light noiseless image sent by the visible light noiseless camera. And forming a pair of visible light images by the received visible light noisy image and the visible light noiseless image.
Step 702: and acquiring a second data set, wherein the second data set comprises at least one pair of infrared light images, and any pair of infrared light images in the second data set comprise a noisy infrared light image and a noiseless infrared light image which are shot on the same object.
In step 702, the same object is captured using an infrared noisy camera and an infrared noiseless camera to obtain a pair of infrared images including a noisy infrared image captured by the infrared noisy camera and a noiseless infrared image captured by the infrared noiseless camera. Wherein, use infrared light noiseless camera and infrared light noiseless camera to shoot different objects, can be to infrared light image more.
The image quality shot by the infrared noisy camera is low, and the noisy infrared light image shot by the infrared noisy camera contains noise. The image quality photographed by the infrared noiseless camera is high, and normally, the noiseless infrared image photographed by the infrared noiseless camera does not contain noise.
In some embodiments, the infrared imaging device or the first device may be connected to an infrared noisy camera and an infrared noiseless camera, respectively. When the infrared noisy camera and the infrared noiseless camera respectively shoot the same object, the infrared imaging device or the first device receives the infrared noisy image sent by the infrared noisy camera and receives the infrared noiseless image sent by the infrared noiseless camera. And forming a pair of infrared light images by the received infrared light noisy image and the infrared light noiseless image.
In some embodiments, the infrared imaging device and the infrared noisy camera may be the same device, such that the infrared imaging device forms a pair of infrared images from an infrared noisy image captured by itself and a received infrared noiseless image.
The execution sequence between the step 701 and the step 702 is not limited to the order, and the step 701 may be executed first and then the step 702 may be executed, or the step 702 may be executed first and then the step 701 may be executed, or the step 701 and the step 702 may be executed simultaneously.
Step 703: the prior knowledge information is obtained based on the first data set and the second data set.
In step 703, a priori knowledge information is obtained based on the a priori knowledge acquisition model, the first data set and the second data set. The method comprises the steps of inputting a first data set and a second data set into a priori knowledge acquisition model, processing the first data set and the second data set by the priori knowledge acquisition model to obtain priori knowledge information, and then acquiring the priori knowledge information output by the priori knowledge acquisition model.
In some embodiments, the infrared camera device obtains the a priori knowledge information, then saves it locally or in a separate storage device, and then uses the a priori knowledge information to perform the method 500 shown in FIG. 5, described above. Or alternatively, the process may be performed,
After the infrared camera device obtains the a priori knowledge information, the a priori knowledge information is sent to the first device, and then the first device performs the method 500 shown in fig. 5 described above using the a priori knowledge information. Or alternatively, the process may be performed,
after the first device obtains the a priori knowledge information, it is stored locally or in a separate storage device, and then uses the a priori knowledge information to perform the method 500 shown in fig. 5, described above. Or alternatively, the process may be performed,
after the first device obtains the prior knowledge information, the prior knowledge information is stored in the infrared imaging device, and then the infrared imaging device performs the method 500 shown in fig. 5 and described above using the prior knowledge information.
Before step 703 is performed, it is further necessary to train an intelligent algorithm based on at least one training sample, resulting in the a priori knowledge acquisition model.
In some embodiments, the process of training the a priori knowledge acquisition model includes operations 7031-7034, respectively 7031-7034.
7031: at least one training sample is obtained.
Any one of the at least one training sample comprises a third data set, a fourth data set and priori knowledge information corresponding to the third data set and the fourth data set, the third data set comprises at least one pair of visible light images, any pair of visible light images in the third data set comprises noisy visible light images and noiseless visible light images shot by the same object, the fourth data set comprises at least one pair of infrared light images, and any pair of infrared light images in the fourth data set comprises noisy infrared light images and noiseless infrared light images shot by the same object.
In some embodiments, the at least one training sample may be downloaded from a network.
In some embodiments, the third data set is obtained in the manner of step 701 and the fourth data set is obtained in the manner of step 702, the a priori knowledge information corresponding to the third data set and the fourth data set is manually analyzed, and the third data set, the fourth data set and the a priori knowledge information are combined into a training sample. The process of obtaining training samples is repeated, and a plurality of training samples are obtained.
7032: and acquiring prior knowledge information corresponding to each training sample based on the prior knowledge acquisition model to be trained and the third data set and the fourth data set in each training sample.
For each training sample, the prior knowledge information corresponding to the training sample is the prior knowledge information output after the to-be-trained prior knowledge acquisition model processes the third data set and the fourth data set in the training sample.
The flow type recognition model to be trained comprises a convolutional neural network, a random forest algorithm, a logistic regression algorithm or a support vector machine (support vector machine, SVM) and the like.
In 7032 operation, a third data set and a fourth data set included in each training sample are input into a priori knowledge acquisition model to be trained, so that the priori knowledge acquisition model to be trained can acquire the priori knowledge information corresponding to each training sample based on the third data set and the fourth data set included in each training sample, and the prior knowledge information corresponding to each training sample output by the prior knowledge acquisition model to be trained can be acquired.
7033: and calculating a loss value through a loss function based on the prior knowledge information included in each training sample and the prior knowledge information corresponding to each training sample, and adjusting parameters of the prior knowledge acquisition model to be trained based on the loss value.
7034: and returning to 7032 operation when the prior knowledge acquisition model to be trained is determined to be continuously trained, and taking the prior knowledge acquisition model to be trained as the prior knowledge acquisition model when the prior knowledge acquisition model to be trained is determined not to be continuously trained.
In some embodiments, when the number of times the prior knowledge acquisition model to be trained reaches a specified number of times, it is determined that the prior knowledge acquisition model to be trained is not to be continued to be trained. Or alternatively, the process may be performed,
And acquiring the correct rate of the prior knowledge acquisition model to be trained by using a plurality of check samples, and determining that the prior knowledge acquisition model to be trained is not to be trained when the correct rate exceeds a specified correct rate threshold. When the method is realized, the following steps are:
a plurality of check samples are acquired, each check sample including a third data set, a fourth data set, and first a priori knowledge information. And acquiring second priori knowledge information corresponding to each check sample based on the to-be-trained priori knowledge acquisition model, the third data set and the fourth data set included in each check sample. And calculating the accuracy of the prior knowledge information based on the first prior knowledge information included in each check sample and the second prior knowledge information corresponding to each check sample. And when the accuracy rate does not exceed the specified accuracy rate threshold, determining to continue training the prior knowledge acquisition model to be trained, and when the accuracy rate exceeds the specified accuracy rate threshold, determining to not continue training the prior knowledge acquisition model to be trained.
In some embodiments, after the infrared imaging device trains the prior knowledge acquisition model, the prior knowledge acquisition model is stored locally, and then the method 700 shown in fig. 7 is performed using the prior knowledge acquisition model when the prior knowledge information needs to be acquired. Or alternatively, the process may be performed,
After the infrared camera device trains the prior knowledge acquisition model, the prior knowledge acquisition model is sent to the first device, and then when the prior knowledge information needs to be acquired, the first device uses the prior knowledge acquisition model to execute the method 700 shown in fig. 7. Or alternatively, the process may be performed,
after the first device trains the prior knowledge acquisition model, the prior knowledge acquisition model is stored locally, and then the method 700 shown in fig. 7 is performed using the prior knowledge acquisition model when the prior knowledge information needs to be acquired. Or alternatively, the process may be performed,
after the first device trains the prior knowledge acquisition model, the prior knowledge acquisition model is saved in the infrared imaging device, and then when the prior knowledge information needs to be acquired, the infrared imaging device uses the prior knowledge acquisition model to execute the method 700 shown in fig. 7. Or alternatively, the process may be performed,
after the third party device other than the first device and the infrared imaging device trains the prior knowledge acquisition model, the prior knowledge acquisition model is stored in the infrared imaging device or the first device, and then when the prior knowledge information needs to be acquired, the infrared imaging device or the first device uses the prior knowledge acquisition model to execute the method 700 shown in fig. 7.
In the embodiment of the application, the priori knowledge information is acquired, so that when the second image needs to be subjected to the second noise elimination, the priori knowledge information is directly used for eliminating the non-uniform noise in the second image, the accuracy of correcting the image is further improved, and the effect of eliminating the non-uniform noise is improved.
Referring to fig. 8, an embodiment of the present application provides an apparatus 800 for processing an image, where the apparatus 800 is disposed on the infrared imaging device 100 of the embodiment shown in fig. 1 or fig. 2, or where the apparatus 800 is disposed on the first device 301 or the infrared imaging device 100 in the network architecture 300 shown in fig. 3, or where the apparatus 800 is disposed on the first device or the infrared imaging device in the method 400 shown in fig. 4, or where the apparatus 800 is disposed on the first device or the infrared imaging device in the method 600 shown in fig. 6, or where the apparatus 800 is disposed on the first device or the infrared imaging device in the method 700 shown in fig. 7. Referring to fig. 8, the apparatus 800 includes.
An acquiring unit 801, configured to acquire a first image and a first temperature, where the first image is an image acquired by an infrared detector, and the first temperature is a temperature of the infrared detector when the first image is acquired;
an acquisition unit 801 for acquiring correction parameters of the first image based on the first temperature;
a processing unit 802, configured to cancel non-uniformity noise included in the first image based on the correction parameter to obtain a second image.
Optionally, the detailed implementation process of the acquiring unit 801 for acquiring the first image and the first temperature is referred to in the relevant content of step 401 of the method 400 shown in fig. 4, which is not described in detail here.
Alternatively, the detailed implementation process of the acquiring unit 801 for acquiring the correction parameters, see the relevant content of step 402 of the method 400 shown in fig. 4, will not be described in detail here.
Optionally, the detailed implementation process of the processing unit 802 to eliminate the non-uniformity noise contained in the first image, see the relevant content of step 403 of the method 400 shown in fig. 4, which is not described in detail here.
Optionally, the correction parameter of the first image includes a correction parameter of a first pixel, where the correction parameter of the first pixel is used to correct non-uniformity noise included in the first pixel, and the first image includes the first pixel;
the obtaining unit 801 is configured to obtain a correction parameter of a first pixel point based on a first correspondence, an index of the first pixel point, and a first temperature, where the first correspondence includes a correspondence between the temperature, the index of the pixel point, and the correction parameter.
Alternatively, the detailed implementation process of the acquiring unit 801 for acquiring the correction parameters, see the relevant content of step 402 of the method 400 shown in fig. 4, will not be described in detail here.
Optionally, the correction parameter of the first pixel point includes a gain coefficient of the first pixel point, and/or a temperature drift coefficient of the first pixel point, where the gain coefficient is used to correct a gain of the first pixel point, and the temperature drift coefficient is used to correct an effect of the first temperature on the first pixel point.
Optionally, the obtaining unit 801 is further configured to:
acquiring M frames of third images and N frames of fourth images, wherein M and N are integers larger than 1, the M frames of third images are images obtained by acquiring a calibration instrument at a third temperature when the infrared detector is at a second temperature, and the N frames of fourth images are images obtained by acquiring a calibration instrument at a fourth temperature when the infrared detector is at the second temperature, and the third temperature is smaller than the fourth temperature;
and acquiring the first corresponding relation based on the M frames of third images and the N frames of fourth images.
Optionally, the detailed implementation process of the acquiring unit 801 to acquire the M-frame third image and the N-frame fourth image is referred to as related content of step 601 and step 602 of the method 600 shown in fig. 6, which will not be described in detail herein.
Optionally, the detailed implementation process of the first correspondence is acquired by the acquiring unit 801, which is referred to in step 603 and step 604 of the method 600 shown in fig. 6, and will not be described in detail herein.
Alternatively, the acquiring unit 801 is configured to:
and acquiring correction parameters of the second pixel point based on M pixel values of the second pixel point and N pixel values of the second pixel point, wherein the M pixel values comprise pixel values of the second pixel point in a third image of each frame, and the N pixel values comprise pixel values of the second pixel point in a fourth image of each frame.
And correspondingly storing the second temperature, the index of the second pixel point and the correction parameter of the second pixel point in the first corresponding relation.
Alternatively, the detailed implementation process of the acquiring unit 801 for acquiring the correction parameters of the second pixel point is referred to as the relevant content of step 603 of the method 600 shown in fig. 6, and will not be described in detail here.
Alternatively, the acquiring unit 801 is configured to:
and acquiring a first pixel value average value based on the M pixel values, and acquiring a second pixel value average value based on the N pixel values, wherein the temperature drift coefficient of the second pixel point is equal to the first pixel value average value.
The first total pixel value average value is obtained based on the pixel value of each pixel point included in the third image of each frame, and the second total pixel value average value is obtained based on the pixel value of each pixel point included in the fourth image of each frame.
And acquiring a gain coefficient of the second pixel point based on the first pixel value average value, the second pixel value average value, the first total pixel value average value and the second total pixel value average value.
Alternatively, the acquiring unit 801 is configured to acquire the first temperature by a temperature sensor.
Optionally, the detailed implementation process of the acquiring unit 801 for acquiring the first temperature through the temperature sensor is referred to as the relevant content of step 401 of the method 400 shown in fig. 4, which is not described in detail herein.
Optionally, the temperature sensor is integrated with the infrared detector.
Optionally, the processing unit 802 is further configured to cancel the non-uniform noise included in the second image to obtain the fifth image based on a priori knowledge information, where the a priori knowledge information is used to reflect a difference between the noisy image and the noiseless image.
Optionally, the detailed implementation process of the processing unit 802 to eliminate the non-uniformity noise contained in the second image, see the relevant content of step 501 and step 502 shown in fig. 5, which will not be described in detail here.
Optionally, the processing unit 502 is configured to:
constructing a noise image based on the prior knowledge information and the second image;
based on the noise image, non-uniformity noise contained in the second image is eliminated to obtain a fifth image.
Optionally, the obtaining unit 801 is further configured to:
acquiring a first data set and a second data set, wherein the first data set comprises at least one pair of visible light images, any pair of visible light images in the first data set comprise noisy visible light images and noiseless visible light images obtained by shooting the same object, the second data set comprises at least one pair of infrared light images, and any pair of infrared light images in the second data set comprise noisy infrared light images and noiseless infrared light images obtained by shooting the same object;
The prior knowledge information is obtained based on the first data set and the second data set.
Alternatively, the detailed implementation procedure of the acquiring unit 801 to acquire the first data set and the second data set is referred to as related content of step 701 and step 702 shown in fig. 7, and will not be described in detail here.
Alternatively, the detailed implementation process of the acquiring unit 801 for acquiring the a priori knowledge information, see the relevant content of step 703 shown in fig. 7, which will not be described in detail here.
Alternatively, the acquiring unit 801 is configured to:
the method comprises the steps of acquiring a priori knowledge information based on a priori knowledge acquisition model, a first data set and a second data set.
Optionally, the obtaining unit 801 is configured to train the intelligent algorithm to obtain a priori knowledge obtaining model based on at least one training sample.
Any one of the at least one training sample comprises a third data set, a fourth data set and priori knowledge information corresponding to the third data set and the fourth data set, the third data set comprises at least one pair of visible light images, any pair of visible light images in the third data set comprises noisy visible light images and noiseless visible light images shot on the same object, the fourth data set comprises at least one pair of infrared light images, and any pair of infrared light images in the fourth data set comprises noisy infrared light images and noiseless infrared light images shot on the same object.
In the embodiment of the application, the acquisition unit acquires a first image and a first temperature, wherein the first image is an image acquired by the infrared detector, the first temperature is the temperature of the infrared detector when the first image is acquired, and the correction parameter of the first image is acquired based on the first temperature. The processing unit eliminates non-uniformity noise contained in the first image based on the correction parameter to obtain a second image. The first temperature is the temperature of the image acquired by the infrared detector, the acquisition unit acquires the correction parameter of the first image based on the first temperature, the correction parameter corresponds to the first temperature, the processing unit eliminates non-uniformity noise contained in the first image based on the correction parameter, the accuracy of correcting the infrared image can be improved, the effect of eliminating the non-uniformity noise in the infrared image is improved, and the non-uniformity noise in the infrared image can be better eliminated, so that the imaging effect of the infrared image is improved.
Referring to fig. 9, an embodiment of the present application provides a schematic diagram of an apparatus 900 for processing an image. The device 900 is the infrared camera device 100 of the embodiment shown in fig. 1 or fig. 2, or the device 900 is the first device 301 or the infrared camera device 100 in the network architecture 300 shown in fig. 3, or the device 900 is the first device or the infrared camera device in the method 400 shown in fig. 4, or the device 900 is the first device or the infrared camera device in the method 600 shown in fig. 6, or the device 900 is the first device or the infrared camera device in the method 700 shown in fig. 7. The device 900 comprises at least one processor 901, an internal connection 902, a memory 903 and at least one transceiver 904.
The apparatus 900 is a device of a hardware structure that may be used to implement the functional modules in the apparatus 700 described in fig. 7. For example, it will be appreciated by those skilled in the art that the acquisition unit 701 and the processing unit 702 in the apparatus 700 shown in fig. 7 may be implemented by the at least one processor 901 invoking code in the memory 903.
Optionally, the apparatus 900 may also be used to implement the first apparatus or the infrared camera apparatus in any of the above embodiments.
Alternatively, the processor 901 may be a general purpose central processing unit (central processing unit, CPU), network processor (network processor, NP), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.
The internal connection 902 may include a path to transfer information between the components. Alternatively, the internal connection 902 is a board or bus, etc.
The transceiver 904 is used to communicate with other devices or communication networks.
The memory 903 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), a compact disc read-only memory (compact disc read-only memory) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be stand alone and coupled to the processor via a bus. The memory may also be integrated with the processor.
The memory 903 is used for storing application program codes for executing the scheme of the present application, and the processor 901 controls the execution. The processor 901 is configured to execute application code stored in the memory 903 and cooperate with at least one transceiver 904 to cause the apparatus 900 to perform the functions of the methods of the present patent.
In a particular implementation, processor 901 may include one or more CPUs, such as CPU0 and CPU1 of FIG. 9, as an embodiment.
In a particular implementation, the device 900 may include multiple processors, such as processor 901 and processor 907 in FIG. 9, as one embodiment. Each of these processors may be a single-core (single-CPU) processor or may be a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the present application is not intended to be limiting, but rather, any modifications, equivalents, improvements, etc. that fall within the principles of the present application are intended to be included within the scope of the present application.
Claims (18)
1. A method of processing an image, the method comprising:
acquiring a first image and a first temperature, wherein the first image is an image acquired by an infrared detector, and the first temperature is the temperature of the infrared detector when acquiring the first image;
acquiring correction parameters of the first image based on the first temperature;
and eliminating non-uniformity noise contained in the first image based on the correction parameters to obtain a second image.
2. The method of claim 1, wherein the correction parameters of the first image include correction parameters of a first pixel for correcting non-uniformity noise contained in the first pixel, the first image including the first pixel;
the acquiring correction parameters of the first image based on the first temperature includes:
and acquiring correction parameters of the first pixel point based on a first corresponding relation, the index of the first pixel point and the first temperature, wherein the first corresponding relation comprises the corresponding relation among the temperature, the index of the pixel point and the correction parameters.
3. The method of claim 2, wherein the correction parameters of the first pixel point include a gain coefficient of the first pixel point and/or a temperature drift coefficient of the first pixel point, the gain coefficient being used to correct a gain of the first pixel point, the temperature drift coefficient being used to correct an effect of the first temperature on the first pixel point.
4. A method according to claim 2 or 3, wherein the method further comprises:
acquiring M frames of third images and N frames of fourth images, wherein M and N are integers larger than 1, the M frames of third images are images obtained by acquiring a calibration instrument at a third temperature when the infrared detector is at a second temperature, the N frames of fourth images are images obtained by acquiring the calibration instrument at a fourth temperature when the infrared detector is at the second temperature, and the third temperature is smaller than the fourth temperature;
and acquiring the first corresponding relation based on the M frames of third images and the N frames of fourth images.
5. The method of any one of claims 1-4, wherein said obtaining a first temperature comprises:
the first temperature is obtained by a temperature sensor.
6. The method as set forth in claim 5 wherein said temperature sensor is integrated with said infrared detector.
7. The method of any one of claims 1-6, wherein the method further comprises:
and eliminating non-uniformity noise contained in the second image to obtain a fifth image based on prior knowledge information, wherein the prior knowledge information is used for reflecting the difference between the noise image and the noiseless image.
8. The method of claim 7, wherein the removing non-uniformity noise contained in the second image based on the a priori knowledge information yields a fifth image, comprising:
constructing a noise image based on the prior knowledge information and the second image;
and eliminating non-uniformity noise contained in the second image based on the noise image to obtain a fifth image.
9. An apparatus for processing an image, the apparatus comprising:
the acquisition unit is used for acquiring a first image and a first temperature, wherein the first image is an image acquired by the infrared detector, and the first temperature is the temperature of the infrared detector when the first image is acquired;
the acquisition unit is further used for acquiring correction parameters of the first image based on the first temperature;
And the processing unit is used for eliminating the non-uniformity noise contained in the first image based on the correction parameters to obtain a second image.
10. The apparatus of claim 9, wherein the correction parameters of the first image comprise correction parameters of a first pixel, the correction parameters of the first pixel being used to correct non-uniformity noise contained in the first pixel, the first image comprising the first pixel;
the obtaining unit is configured to obtain a correction parameter of the first pixel point based on a first correspondence, an index of the first pixel point, and the first temperature, where the first correspondence includes a correspondence among the temperature, the index of the pixel point, and the correction parameter.
11. The apparatus of claim 10, wherein the correction parameters for the first pixel point comprise a gain coefficient for the first pixel point and/or a temperature drift coefficient for the first pixel point, the gain coefficient being used to correct for gain of the first pixel point, the temperature drift coefficient being used to correct for an effect of the first temperature on the first pixel point.
12. The apparatus of claim 10 or 11, wherein the acquisition unit is further configured to:
Acquiring M frames of third images and N frames of fourth images, wherein M and N are integers larger than 1, the M frames of third images are images obtained by acquiring a calibration instrument at a third temperature when the infrared detector is at a second temperature, the N frames of fourth images are images obtained by acquiring the calibration instrument at a fourth temperature when the infrared detector is at the second temperature, and the third temperature is smaller than the fourth temperature;
and acquiring the first corresponding relation based on the M frames of third images and the N frames of fourth images.
13. The apparatus according to any one of claims 9-12, wherein the acquisition unit is configured to acquire the first temperature by means of a temperature sensor.
14. The apparatus of claim 13 wherein said temperature sensor is integrated with said infrared detector.
15. The apparatus of any one of claim 9 to 14,
the processing unit is further configured to cancel non-uniform noise included in the second image to obtain a fifth image based on a priori knowledge information, where the a priori knowledge information is used to reflect a difference between a noise image and a noise-free image.
16. The apparatus of claim 15, wherein the processing unit is to:
constructing a noise image based on the prior knowledge information and the second image;
and eliminating non-uniformity noise contained in the second image based on the noise image to obtain a fifth image.
17. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a computer, implements the method according to any of claims 1-8.
18. A computer program product, characterized in that the computer program product comprises a computer program stored in a computer readable storage medium and that the computer program is loaded by a computer to implement the method according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210137854.9A CN116645310A (en) | 2022-02-15 | 2022-02-15 | Method, device and storage medium for processing image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210137854.9A CN116645310A (en) | 2022-02-15 | 2022-02-15 | Method, device and storage medium for processing image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116645310A true CN116645310A (en) | 2023-08-25 |
Family
ID=87617534
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210137854.9A Pending CN116645310A (en) | 2022-02-15 | 2022-02-15 | Method, device and storage medium for processing image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116645310A (en) |
-
2022
- 2022-02-15 CN CN202210137854.9A patent/CN116645310A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7443366B2 (en) | Artificial intelligence techniques for image enhancement | |
| US7016550B2 (en) | Scene-based non-uniformity offset correction for staring arrays | |
| US6901173B2 (en) | Scene-based non-uniformity correction for detector arrays | |
| CN110166704B (en) | Calibration method and device of multispectral camera | |
| US8203116B2 (en) | Scene based non-uniformity correction for infrared detector arrays | |
| US8428385B2 (en) | Non-uniformity error correction with a bilateral filter | |
| EP2721828B1 (en) | High resolution multispectral image capture | |
| US20100220193A1 (en) | Systems and methods for processing infrared images | |
| KR101955498B1 (en) | Infrared image correction apparatus using neural network structure and method thereof | |
| CN106488201A (en) | A kind of processing method of picture signal and system | |
| EP2228677A1 (en) | Extended depth-of-field surveillance imaging system | |
| JP2014521263A (en) | Automatic adaptive image sharpening | |
| CN112672054B (en) | Focusing method and device and electronic equipment | |
| CN110121031A (en) | Image-pickup method and device, electronic equipment, computer readable storage medium | |
| JP7127647B2 (en) | IMAGING DEVICE, IMAGING METHOD, AND PROGRAM | |
| CN113432723A (en) | Image processing method, system and computer system for weakening stray radiation | |
| CN114581377A (en) | Danger detection method and device | |
| CN116645310A (en) | Method, device and storage medium for processing image | |
| CN117218026A (en) | Infrared image enhancement method and device | |
| WO2021127972A1 (en) | Image processing method and apparatus, imaging device, and movable carrier | |
| CN116668831A (en) | Consistency adjusting method and device for multi-camera system | |
| CN116543022A (en) | Gas infrared image processing method, gas leak detector and storage medium | |
| Yu et al. | An improved retina-like nonuniformity correction for infrared focal-plane array | |
| WO2020107291A1 (en) | Photographing method and apparatus, and unmanned aerial vehicle | |
| CN110567585A (en) | Real-time infrared image pot cover effect inhibition method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |