CN112399164A - Color restoration method for color image containing near infrared rays under low illumination condition - Google Patents
Color restoration method for color image containing near infrared rays under low illumination condition Download PDFInfo
- Publication number
- CN112399164A CN112399164A CN202011101487.4A CN202011101487A CN112399164A CN 112399164 A CN112399164 A CN 112399164A CN 202011101487 A CN202011101487 A CN 202011101487A CN 112399164 A CN112399164 A CN 112399164A
- Authority
- CN
- China
- Prior art keywords
- color
- near infrared
- color image
- image
- low illumination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000005286 illumination Methods 0.000 title claims abstract description 34
- 238000012549 training Methods 0.000 claims abstract description 27
- 239000011159 matrix material Substances 0.000 claims abstract description 18
- 230000009466 transformation Effects 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims abstract description 9
- 230000009467 reduction Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims abstract description 6
- 238000005457 optimization Methods 0.000 claims description 2
- 238000011084 recovery Methods 0.000 abstract description 3
- 238000011160 research Methods 0.000 description 14
- 238000003384 imaging method Methods 0.000 description 9
- 238000001914 filtration Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 238000012937 correction Methods 0.000 description 5
- 238000001228 spectrum Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 238000011946 reduction process Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 229910052710 silicon Inorganic materials 0.000 description 2
- 239000010703 silicon Substances 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 101100248200 Arabidopsis thaliana RGGB gene Proteins 0.000 description 1
- PEDCQBHIVMGVHV-UHFFFAOYSA-N Glycerine Chemical compound OCC(O)CO PEDCQBHIVMGVHV-UHFFFAOYSA-N 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000003702 image correction Methods 0.000 description 1
- 238000003331 infrared imaging Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 239000012780 transparent material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/88—Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/30—Transforming light or analogous information into electric information
- H04N5/33—Transforming infrared radiation
Abstract
The invention discloses a color restoration method for a color image containing near infrared under a low illumination condition, which is a method for improving the quality of an acquired true color image under the low illumination condition by utilizing a common color CMOS/CCD. The method comprises a training and recovery process, wherein the training comprises four steps: under the condition of low illumination, 2 ColorChecker standard 24-color card images containing or not containing near infrared are obtained by using CMOS/CCD to open and close a 'hot mirror' respectively for shooting; respectively carrying out noise reduction processing on the images; respectively solving the average RGB value of each color block in the image to obtain RGB training data pairs containing or not containing near infrared; and obtaining a transformation matrix from the image containing the near infrared to the true color image by using the data pairs in a least square mode. The restoration process is to transform the near infrared image into true color image via the transformation matrix. The invention only needs a ColorChecker standard 24-color card; near-infrared auxiliary lighting is not required to be added; the method is a method for simply improving the quality of the true color image and the video under the low illumination condition by the algorithm.
Description
Technical Field
The invention provides a color restoration method for a color image containing near infrared under a low illumination condition, belongs to the technical field of color and digital imaging, and relates to a method for improving the quality of a true color image and a video under the low illumination condition.
Background
Capturing color images with CCD/CMOS under low light conditions has been a difficult problem, and noise reduction and deblurring of color images captured under low light conditions has been very difficult[1]. The general camera adopts two methods, one is a visible light flash lamp, and the other is a method for prolonging the exposure time. Shadow and specular reflection can be generated by using a flash lamp to influence color acquisition, strong noise and motion blur can be generated in a shot image simultaneously due to long-time exposure, and the frame frequency can be reduced for acquiring a video image. Researchers have proposed many sophisticated image denoising[2]And a deblurring method[3]But the difficulty is large and the effect is not ideal. Under normal lighting conditions, the imaging system removes Near Infrared (NIR) using an infrared light cut filter (known as a "hot-mirror") placed in front of the sensor. In low light conditions, the "hot mirror" is typically removed and NIR-assisted illumination is added in order to improve the signal-to-noise ratio. The NIR light source spectrum is completely different from the visible light spectrum, so that real color acquisition of a scene cannot be realized at night, and a black and white mode is adopted at the moment. The video surveillance system has a "day/night" color/black-and-white switching mode. How to improve the quality of true color images under low illumination by using NIR wave band becomes important research content.
In 2009, the Hertel team at Cambridge university, UK studied the high dynamic range conditions under different lighting environments at night[1]Color correction for NIR-containing color images and grayscale images by eliminating "hot mirrorsThe problem is positive. The color image R 'G' B 'collected after the hot mirror is removed contains NIR, and the NIR signal portion must be eliminated from the R' G 'B' signal, which in principle provides an estimate of the NIR portion of the incident light. Matrix elements are obtained by taking a Macbeth color chart as a training sample. The experimental results show that the color restoration effect is good for color cards and nearby colors and similar lighting conditions, but the color restoration effect is poor for different objects under a night lighting light source, and particularly, the color of a self-luminous car lamp which is not a reflection color is poor, and the brightness of the car lamp can be far beyond the set range. In 2009 and 2011, Susstrun research team, Switzerland studied the elimination of "hot mirrors" in traditional sensor arrays "[4]A filter for filtering visible light is added, visible light and NIR images of the same scene are continuously shot twice, and a demosaicing interpolation algorithm is designed for estimating visible information and NIR information. The correlation matrix and the corresponding CFA and demosaicing matrices are calculated using 25 images as a training set, the remaining images being used to test the validity of the obtained CFA and demosaicing matrices. In 2013, the japanese Takeuchi research team proposed a method of composite color imaging using a noisy color image and a NIR image of a near-infrared auxiliary light source[5]. The method is that the luminance component and the chrominance component of the synthesized color image are estimated using different image sources. The luminance component is estimated from the flash image by a spectral estimation method, and the chrominance component is estimated after denoising from the noise-containing color image. The proposed method is superior to simple color image noise reduction and also superior to existing noise color images and gray scale images taken by near infrared flash. In 2015, the Sugimura team of Tokyo university proposed an imaging system and method for obtaining color image sequences under extremely low light conditions[6, 7]. The system simultaneously acquires a near infrared gray scale image (NIR) and a visible color image (RGB), respectively. RGB employs long exposure times to obtain sufficient color information, and NIR employs short exposure times to obtain scene structure information. And reconstructing a noise-reduction, deblurring and clear color image sequence by using the captured pair of the same scene images and adopting an adaptive smoothing algorithm based on gradient and color correlation.
Near Infrared (NIR) is one of the closest regions to the wavelength of radiation detectable by the human eye, unlike the human eye, silicon-based image sensors CCD/CMOS, which are sensitive to NIR wavelengths, typical CMOS spectral responses are maximal between the wavelengths 600 and 700 nm, extending from visible to up to 1100 nm. The image sensor is monochromatic in nature, and in order to obtain color information, a mosaic filter array (CFA) is placed in front of the sensor, made up of three cone cell red (R), green (G), and blue (B) transparent materials simulating the human eye, which are also generally transparent to NIR. The color of the picture shot by the image sensor is unnatural, namely, the saturation is low and the picture is pink.
By utilizing the inherent characteristic of the response of the CCD/CMOS device to the NIR waveband, the improvement of the color imaging quality under the conditions of sensitivity, low illumination level and low illumination becomes an important research direction. In order to improve sensitivity, visible light CMOS sensor technology using NIR is continuously developed. In the manufacture of color image sensors, filter array patterns (CFAs) are constantly being improved, and RGBW is proposed from the conventional Bayer (Bayer) CFA pattern RGGB[8]、RGBI[9]Sparse bayer, etc[10]In particular, in recent years, a single sensor is proposed to simultaneously acquire RGB-NIR modes of a visible and NIR multiband filter array[11]。
In recent years, research on simultaneous acquisition of visible and near-infrared RGBN by a single sensor has been conducted. In 2014, research team of university of arizona in usa proposed an image acquisition system for simultaneously acquiring RGB and NIR[12]Only white balance algorithms, color adaptation and camera color characterization are employed for NIR containing color image correction. The use of NIR in RGB image enhancement, subtraction, enhancement of blood vessels is being intensively studied. 2016, Park and Kang of Korea, proposed a multispectral image sensor filter pattern RGBN[13]Demosaicing and color correction were studied. In 2016, Teraraka et al, Japan proposed a single sensor to acquire both RGB and NIR filter arrays, and studied the CFA array, demosaicing, and color correction for optimal performance[11]. In particular, two types of filter arrays and demosaicing algorithms are proposed for improving the quality of the acquired RGB and NIR images. In 2017, the Yamashita research team in Japan designed a single CMOSCFA (computational fluid dynamics) for simultaneously acquiring R + NIR, G + NIR, B + NIR and NIR raw data by using sensor[14]And a dual-wavelength band-pass filter is added for filtering
Incident light in a wavelength range. Experimentally using an imaging system at about light intensity
20 raw data were captured in a low light scene.
In summary, the characteristic of the silicon-based image sensor CCD/CMOS that is sensitive to Near Infrared (NIR) below the 1100 nm spectrum, and the use of NIR to improve the imaging quality becomes an important research content of the sensor, which currently includes two aspects of research: on one hand, on the basis of improvement of the traditional color filtering mode RGB, the NIR multispectral filtering mode RGBN is added to the single sensor. The mosaic color filter mode requires interpolation algorithm demosaicing and color correction, and the design of the RGBN mode and demosaicing and color correction are in the early stage of research. In addition, the RGBN mode increases the NIR spatial position, and reduces the spatial resolution and brightness of the color RGB image, which is not mature at present. The color filter pattern is critical to image quality and cannot be easily modified. On the other hand, application research on enhancement, fusion, defogging, denoising, shadow detection, face detection and the like of visible light color (RGB) or gray level images by using NIR is developed in the fields of computer vision and image processing, so that a good effect is achieved, but research on improvement of true color imaging quality under the condition of low-illumination is less, and further research and exploration are needed in theory and method. The related algorithms for improving the effect of color images in low light conditions in favor of images containing NIR gray scale require further research.
In order to meet the application requirements, the invention provides a true color video acquisition method which is simple in algorithm, suitable for common color CMOS/CCD and good in quality of acquired true color video under the condition of low illumination.
Disclosure of Invention
The invention discloses a color restoration method for a color image containing near infrared rays under a low illumination condition, which can improve the quality of true color of the image under the low illumination condition.
The specific technical scheme is as follows: a color restoration method for a color image containing near infrared under low illumination conditions comprises a training and restoration process, wherein the training comprises the following steps: 2 ColorChecker standard 24 color card images containing and not containing near infrared bands are obtained by opening and closing a 'hot mirror' to shoot through a common color CMOS/CCD under the condition of low illumination, namely a 'color image' containing the near infrared bands and a true color image only containing visible light bands are respectively obtained; carrying out mean value filtering and noise reduction processing on the 2 images respectively; respectively solving the average RGB value of each color of the 24-color card of each image to obtain RGB data pairs containing and not containing near-infrared bands as training data pairs; obtaining a transformation matrix from a true color image containing a near infrared band to a true color image not containing the near infrared by using a data pair in a least square mode; when the restoration process is actually applied, the shot image containing the near infrared band is converted into a true color image without near infrared through a transformation matrix.
Further, turning on and off the "hot mirror" is a switch function provided by an infrared cut filter (IR-cut filter, referred to as a "hot-mirror") itself used in a common day-type CMOS/CCD camera.
Further, the training sample data pair is a "color image" in the near infrared band and a corresponding value of the 24 color blocks RGB in the true color image only containing the visible light band.
Further, a transformation matrix from the color image containing the near infrared band to the true color image is obtained by adopting a least square method optimization algorithm through a ColorChecker standard 24 color card training sample data pair.
Further, for color images containing near infrared band
Indicating that RGB of the true color restoration image is realized by the transformation matrix a as follows:
the method for improving the true color quality of the image under the condition of low illumination by using the common CMOS/CCD only needs a ColorChecker standard 24 color card used for correcting the color of a common camera; near-infrared auxiliary lighting is not required to be added; the method is simple in algorithm and capable of improving the quality of the true color image and the video under the low illumination condition.
The invention uses common color CMOS/CCD to shoot training data pairs with and without near infrared band when the hot mirror is opened and closed under low illumination condition, to obtain the transformation matrix from color image with near infrared band to true color image.
Drawings
FIG. 1. training steps and calibration flow chart included in the method of the present invention;
FIG. 2 ColorChecker Standard 24 color card;
FIG. 3 is a diagram of a color chart with near infrared color images taken in the "hot mirror" on state;
FIG. 4 is a color image of a color chart taken in an "off" state of a "hot mirror" switch;
FIG. 5 illustrates a noise reduction process for a color chart containing near infrared color images taken in the "hot mirror" on "state;
FIG. 6 illustrates a color image noise reduction process for taking a color chart in the "hot mirror" on "state;
FIG. 7 is a diagram of a near infrared-containing color image of an actual scene taken in the "hot mirror" on "state;
FIG. 8 is a view for taking a color image of an actual subject in an "off" state of the "hot mirror" switch;
fig. 9 shows the actual scene image after being processed by the invented method.
Detailed Description
The technical scheme of the invention is described in detail in the following with reference to the attached drawings:
the method comprises training and recovery, wherein the training comprises the following 4 steps:
step 1: 2 ColorChecker standard 24 color card images with and without near infrared bands were taken with a common color CMOS/CCD on and off the "hot mirror" under low light conditions, respectively.
Under normal lighting conditions, a common color CMOS/CCD removes Near Infrared (NIR) using an infrared light cut-off filter (hot mirror) placed in front of the sensor. In low light conditions, the "hot mirror" is typically removed and NIR-assisted illumination is added in order to improve the signal-to-noise ratio. The NIR light source spectrum is completely different from the visible light spectrum, so that real color acquisition of a scene cannot be realized at night, and a black and white mode is adopted at the moment. For example, the video surveillance system has a "color/black-and-white" switching mode for day/night.
The method comprises opening and closing a hot mirror to shoot ColorChecker standard 24 color card (as shown in FIG. 1) containing and not containing near infrared band under low illumination to obtain two color images, which are color images containing and not containing near infrared band "
And a true color image RGB containing only visible light. The low light conditions include moonlight, bright starlight conditions. The ColorChecker standard 24 color card is shown in fig. 2.
Step 2: and respectively carrying out denoising treatment on the images containing and not containing the near infrared band.
Each color on the ColorChecker Standard 24 color card is uniform, but contains a "color image" of the near infrared band under low illumination conditions "
And the true color image RGB containing only visible light contains noise, resulting in large RGB fluctuation of each color patch, and for this reason, noise reduction processing is performed. The image blurring degree after noise reduction does not need to be considered particularly, and a common mean value filtering method is adopted.
And 3, step 3: the RGB values of the 24 color chips are taken as training data pairs.
For color images containing near infrared wave bands "
And after noise reduction processing is carried out on the real color image RGB only containing visible light, the average value of all pixels RGB three channels of each color block is obtained to obtain a color image containing a near infrared band "
And data of color blocks corresponding to two images of the true color image RGB only containing visible light are shown in Table 1. When the average value of all the RGB three channels of the pixels of each color block is calculated, the pixels with certain width at the edges of the color blocks can be removed, and the influence of the edges of the adjacent color blocks is eliminated.
And 4, step 4: obtaining images from a camera containing near infrared bands by using training data pairs
To a true color image RGB without near infrared.
Obtaining an image containing a near infrared band by using a training data pair and adopting a least square method
Transformation matrix a to true color image RGB without near infrared:
the recovery process comprises the following steps: the true color image from near infrared band to no near infrared band is converted by the transformation matrix in practical application.
Obtaining an image containing a near infrared band from an actual shooting scene by using the color space transformation matrix A obtained in the 4 th step of the training process
To a true color image RGB without near infrared.
Examples
The experimental device comprises: (1) low-illumination CMOS: the Sony Semiconductors CMOS is IMX385LQR, and the pixel size is 3.75
Diagonal 8.35 mm (1/2 inches); video format: 1080P maximum resolution 1920
1080; for the night environment of micro light and starlight, the excellent ultra-low illumination image quality is in the environment of ultra-low lux illumination. (2) A drive circuit: the CMOS is self-contained; the circuit provides a control circuit that can control both the "hot mirror" switch "on" and "off. (3) Optical objective lens: industrial IR lens: target surface size 2/3 inches; the focal length is 16 mm; compatible with VIS and NIR band broad spectrum 350-1100 nm (4) color card: U.S. X-Rite ColorChecker Standard 24 color card.
The experimental shooting condition is that under the natural night full moon condition, the brightness measured by a brightness instrument is 0.0026 lux.
The training images are shot to shoot the color chart under two states of 'hot mirror' switch 'on' and 'off' to obtain a color image containing near infrared and a true color image under the condition of low illumination, as shown in fig. 3 and 4.
Denoising a training sample, and extracting training data: color card image sampling 5 containing near-infrared color image and true color image under low illumination condition
The 5 mean filtering algorithm denoises, and the denoised image is shown in fig. 5 and 6. And taking out 24 color blocks to obtain corresponding training data.
Shooting a test image: the actual scene is shot in two states of 'hot mirror' switch 'on' and 'off' to obtain a color image containing near infrared rays and a true color image under the condition of low illumination, as shown in fig. 7 and 8.
The calculated transformation matrix elements are shown in table 2.
The restoration effect test is to restore, i.e. matrix transform, a color image containing near infrared rays of an actual scene taken in the state of switching "on" to the "hot mirror", and the result is shown in fig. 9. Subjective and objective assessment of vision can be derived: (1) the pink color cast of the color image containing near infrared rays is improved; (2) compared with the color image of the actual scene shot under the state of 'hot mirror' switch 'off', the color is recovered, and the details are increased; (3) the color image signal ratio of the actual scene is 21.12, the actual scene contains 25.08 of the signal-to-noise ratio after the near-infrared color image is restored, and the signal-to-noise ratio is improved by 18.75 percent.
Corresponding references to the background art
[1] D. Hertel, H. Marechal, D. Tefera, W. Fan, R. Hicks, A low-cost VIS-NIR true color night vision video system based on a wide dynamic range CMOS imager, IEEE Proc. Intelligent Vehicles Symposium, 273 – 278, 2009.
[2] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3D transform-domain collaborative filtering, IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080–2095, Aug. 2007.
[3] Z. Hu, S. Cho, J. Wang, and M.-H. Yang, Deblurring low-light images with light streaks, Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 3382–3389, Jun. 2014.
[4] Zahra Sadeghipoor, Yue M. Lu, and Sabine Susstrunk, Correlation-based joint acquisition and demosaicing of visible and near-infrared images, IEEE International Conference on Image Processing, 6626 (1) : 3165-3168, 2011.
[5] K. Takeuchi, M. Tanaka, M. Okutomi, Low-Light Scene Color Imaging Based on Luminance Estimation from Near-Infraraed Flash Image, IEEE International Workshop on Computational Camera and Displays, June, 2013.
[6] T. Mikami, D. Sugimura, and T. Hamamoto, Capturing color and near-infrared images with different exposure times for image enhancement under extremely low-light scene, Proc. IEEE Int. Conf. Image Process., pp. 669–673, Oct. 2014.
[7] Daisuke Sugimura, Takuya Mikami, Hiroki Yamashita, and Takayuki Hamamoto, Enhancing Color Images of Extremely Low Light Scenes Based on RGBNIR Images Acquisition With Different Exposure Times, IEEE TRANSACTIONS ON IMAGE PROCESSING, 24(11): 3586-3597, 2015.
[8] H. Honda, Y. Iida, G. Itoh, Y. Egawa, H. Seki, A novel Bayer-like WRGB color filter array for CMOS image sensors," Proc. SPIE-IS&T Electronic Imaging, Vol. 6492, pp. 64921J-1-10, 2007.
[9]S. Huang, J. P. Giacalone, W. Fan, A RGBI Imager Based Background Suppression Method, Proc. IEEE Intelligent Vehicles, 643-647, 2008.
[10]D. P. Haefner, J. P. Reynolds, J. Cha, and V. Hodgkin, Modeling Human Performance with Low Light Sparse Color Imagers, Infrared Imaging Systems: Design, Analysis, Modeling, and Testing XXII, Proc. SPIE Vol. 8014, 2011.
[11] Hayato Teranaka, Yusuke Monno, Masayuki Tanaka and Masatoshi, Single-Sensor RGB and NIR Image Acquisition Toward Optimal Performance by Taking Account of CFA Pattern, Demosaicking, and Color Correction, Electronic Imaging, (18) :1-6, 2016.
[12]Z. Chen, X.Wang, and R. Liang, RGB-NIR multispectral camera, Opt. Express, vol. 22(5): 4985-4994, 2014.
[13]Chulhee Park and Moon Gi Kang, Color Restoration of RGBN Multispectral Filter Array Sensor Images Based on Spectral Decomposition, Sensors, 16, 719, 2016.
[14]Hiroki Yamashita, Daisuke Sugimura, Takayuki Hamamoto, Low-light color image enhancement via iterative noise reduction using RGB/NIR sensor, Journal of Electronic Imaging, 26(4): 043017-1-14, 2017.
Claims (5)
1. A color restoration method for a color image containing near infrared under a low illumination condition is characterized by comprising a training process and a restoration process, wherein the training process comprises the following steps: 2 ColorChecker standard 24 color card images containing and not containing near infrared bands are obtained by opening and closing a 'hot mirror' to shoot through a common color CMOS/CCD under the condition of low illumination, namely a 'color image' containing the near infrared bands and a true color image only containing visible light bands are respectively obtained; respectively carrying out noise reduction processing on the 2 images; respectively solving the average RGB value of each color of the 24-color card of each image to obtain RGB data pairs containing and not containing near-infrared wave bands as training data pairs; obtaining a transformation matrix of a true color image from a true color image containing a near infrared band to a true color image not containing the near infrared by using the data pairs; when the restoration process is actually applied, the shot image containing the near infrared band is converted into a true color image without near infrared through a transformation matrix.
2. The method for color restoration of a color image containing near infrared rays under low illumination according to claim 1, wherein the turning on and off of the "hot mirror" is performed by using a switching function of an infrared cut filter itself used in a common day-type CMOS/CCD camera.
3. The method according to claim 1, wherein the training sample data pairs are color images in the near-infrared band and corresponding values of RGB of 24 color patches in a true color image only containing the visible light band.
4. The method for color restoration of a color image containing near infrared under a low illumination condition according to claim 1, wherein the transformation matrix from the color image containing the near infrared band to the true color image is obtained by using a least square optimization algorithm through a ColorChecker standard 24 color card training sample data pair.
5. The low-light bar of claim 1The method for restoring the color of the under-file color image containing near infrared is characterized in that the color image containing the near infrared band is represented, and the RGB of the true color restoration image is realized by a transformation matrix A, and the method comprises the following steps:
。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011101487.4A CN112399164A (en) | 2020-10-15 | 2020-10-15 | Color restoration method for color image containing near infrared rays under low illumination condition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011101487.4A CN112399164A (en) | 2020-10-15 | 2020-10-15 | Color restoration method for color image containing near infrared rays under low illumination condition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112399164A true CN112399164A (en) | 2021-02-23 |
Family
ID=74596564
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011101487.4A Pending CN112399164A (en) | 2020-10-15 | 2020-10-15 | Color restoration method for color image containing near infrared rays under low illumination condition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112399164A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040239784A1 (en) * | 2003-03-25 | 2004-12-02 | Fuji Photo Film Co., Ltd. | Color-image pickup device in which an R picture signal is relatively enhanced with distance from center of light-reception area |
| CN101202845A (en) * | 2007-11-14 | 2008-06-18 | 北京大学 | Method for changing infrared image into visible light image |
| CN102238306A (en) * | 2010-04-26 | 2011-11-09 | 夏普株式会社 | Image reading apparatus, image data output processing apparatus, and image reading method |
| CN103686111A (en) * | 2013-12-31 | 2014-03-26 | 上海富瀚微电子有限公司 | Method and device for correcting color based on RGBIR (red, green and blue, infra red) image sensor |
-
2020
- 2020-10-15 CN CN202011101487.4A patent/CN112399164A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040239784A1 (en) * | 2003-03-25 | 2004-12-02 | Fuji Photo Film Co., Ltd. | Color-image pickup device in which an R picture signal is relatively enhanced with distance from center of light-reception area |
| CN101202845A (en) * | 2007-11-14 | 2008-06-18 | 北京大学 | Method for changing infrared image into visible light image |
| CN102238306A (en) * | 2010-04-26 | 2011-11-09 | 夏普株式会社 | Image reading apparatus, image data output processing apparatus, and image reading method |
| CN103686111A (en) * | 2013-12-31 | 2014-03-26 | 上海富瀚微电子有限公司 | Method and device for correcting color based on RGBIR (red, green and blue, infra red) image sensor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Namer et al. | Advanced visibility improvement based on polarization filtered images | |
| US11622085B2 (en) | Multispectral image decorrelation method and system | |
| US9979941B2 (en) | Imaging system using a lens unit with longitudinal chromatic aberrations and method of operating | |
| Martinello et al. | Dual aperture photography: Image and depth from a mobile camera | |
| KR20210079282A (en) | Correcting photo underexposure using a neural network | |
| US9787915B2 (en) | Method and apparatus for multi-spectral imaging | |
| CN109804619A (en) | Image processing apparatus, image processing method and camera | |
| CN113676628B (en) | Image forming apparatus and image processing method | |
| CN108712608A (en) | Terminal device image pickup method and device | |
| US10726531B2 (en) | Resolution enhancement of color images | |
| CN113676629B (en) | Image sensor, image acquisition device, image processing method and image processor | |
| Wang et al. | Stereoscopic dark flash for low-light photography | |
| CN112104847B (en) | SONY-RGBW array color reconstruction method based on residual error and high-frequency replacement | |
| JP2011176710A (en) | Imaging apparatus | |
| Haneishi et al. | Image acquisition technique for high dynamic range scenes using a multiband camera | |
| CN112399164A (en) | Color restoration method for color image containing near infrared rays under low illumination condition | |
| JP2020120204A (en) | Image processing device, image processing method, and program | |
| KR101947097B1 (en) | Image Signal Processor for controlling the total shutter image sensor module on the stroboscope | |
| Park et al. | Low light imaging system with? Expanding spectrum band for digital camera | |
| Monno et al. | Direct spatio-spectral datacube reconstruction from raw data using a spatially adaptive spatio-spectral basis | |
| Hardeberg et al. | Multispectral colour imaging: Time to move out of the lab? | |
| JP2013187711A (en) | Image processing apparatus, imaging apparatus, and image processing method | |
| Konnik et al. | Optical-digital correlator with increased dynamic range using spatially varying pixels exposure technique | |
| Ravshanov et al. | Image contrast enhancement based on infrared data obtained under difficult light conditions | |
| Liu et al. | Retinex-Based Low-Light Hyperspectral Restoration Using Camera Response Model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210223 |