CN110580684A - image enhancement method based on black-white-color binocular camera - Google Patents
image enhancement method based on black-white-color binocular camera Download PDFInfo
- Publication number
- CN110580684A CN110580684A CN201810591239.9A CN201810591239A CN110580684A CN 110580684 A CN110580684 A CN 110580684A CN 201810591239 A CN201810591239 A CN 201810591239A CN 110580684 A CN110580684 A CN 110580684A
- Authority
- CN
- China
- Prior art keywords
- image
- color
- black
- white
- binocular camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 239000011159 matrix material Substances 0.000 claims abstract description 17
- 150000001875 compounds Chemical class 0.000 claims description 8
- 230000001737 promoting effect Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000004040 coloring Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000002474 experimental method Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 241000022852 Letis Species 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
the invention discloses an image enhancement method based on a black-white-color binocular camera, which converts a multispectral image enhancement problem into a low-rank matrix filling problem. Aiming at the problems that time complexity is high and practical use is difficult due to accurate stereo matching, an image enhancement method based on a black-white-color camera is provided, black-white image feature points and a color image are matched, the color values of the matching points in the color image are regarded as the color values of the black-white image feature points, and the color of the black-white image is recovered through the pixel points with known color values by using a low-rank matrix filling method. The method can effectively improve the operation efficiency, improves the image sharpness, enhances the image details, inhibits the image noise and has very important significance in the field of image processing.
Description
the technical field is as follows:
The invention mainly relates to the field of digital image processing, in particular to an image enhancement method based on a black-white-color binocular camera.
background art:
compared with a monocular camera, the binocular camera can break through the physical limitations of optics and sensors, and reconstructs an image with better image quality and higher image resolution through redundant complementary information between two images, thereby obtaining more scene information. With these advantages, the binocular camera shows a tendency to dominate the market instead of the monocular camera.
common binocular camera systems can be broadly divided into two types, one consisting of two color cameras of different focal lengths, and the other consisting of two cameras in different domains, color, near infrared, or black and white. Since a conventional CCD color camera uses a Color Filter Array (CFA) (as shown in fig. 1), an image degradation process such as blurring and noise amplification occurs in an acquired image. Therefore, the color binocular camera system often needs preprocessing such as image denoising and image deblurring, and the problem is an ill-posed problem and is difficult to accurately solve. In contrast to color cameras, black and white cameras have no CFA (as shown in fig. 2), and their sensors can receive all incident light. When receiving light rays of different spectrums (as shown in fig. 3), the receiving efficiency of the black-and-white camera under each spectrum is significantly higher than that of the color camera, so that the quality of the acquired image is higher and the details are richer. The camera system formed by combining the black-and-white camera and the color camera has the advantages of both the black-and-white camera and the color camera, so that the acquired image has the advantages of high sharpness and low noise of the black-and-white camera and has the color, and the defects of the black-and-white camera and the color camera are effectively overcome.
in addition, the image enhancement method based on the black-and-white-color camera generally needs to match a black-and-white image and a color image, and then fuses the two images according to a matching result to obtain the color image with improved sharpness and resolution and suppressed noise. Generally, the black-and-white image and the color image can estimate the parallax between the images by a stereo matching method. The stereo matching method firstly roughly estimates the parallax between images, and then gradually optimizes and adjusts to estimate the final parallax. The estimation result of the method is very accurate, but the time complexity is extremely high, and the method is difficult to be practically applied.
The invention content is as follows:
The technical problem to be solved by the invention is as follows: the invention provides an image enhancement method based on a black-white-color binocular camera, aiming at the problems that the precision parallax estimation time is high in complexity and difficult to be practically applied to processing black-white-color images. The image matching method used by the invention only selects partial characteristic points to carry out simple image matching, then assigns the color information of the characteristic points to corresponding points in the black-and-white image, and finally fuses the black-and-white image and the color image by using a low-rank matrix filling method. The method avoids a complex stereo matching method, and uses a simple matching method based on the feature points, so that the calculation time can be effectively reduced.
In order to solve the technical problems, the technical scheme provided by the invention is as follows:
An image enhancement method based on a black-and-white-color camera is characterized in that binocular image enhancement problems of different modes are converted into a low-rank matrix filling problem, and can be expressed as:
（1）
in the formula (I), the compound is shown in the specification,representing a color image to be sought after,a luminance fidelity term is represented by a number of luminance fidelity terms,a color fidelity term is represented by a color fidelity term,a regularization term is represented as a function of,the representation is made of a black-and-white image,the color value of the corresponding point of the black-and-white image in the color image is represented, the brightness fidelity item is used for promoting the recovered image brightness to approach the pixel value of the black-and-white image, the color fidelity item is used for promoting the recovered image color to approach the color value of the corresponding point of the black-and-white image in the color image, and the brightness fidelity item and the color fidelity item used in the patent are respectively as follows:
（2）
（3）
in the formula (I), the compound is shown in the specification,Indicating the pixel position of the corresponding point in the color image of the black-and-white image,representing a linear transformation of a color image into a black-and-white image,indicates the norm (Frobenius norm, used in this example, is notedBut is not limited to such). The algorithm comprises the following steps:
the method comprises the following steps: acquiring a group of black-white-color images by using a black-white-color binocular camera system;
step two: selecting proper characteristic points in a black and white image, searching an image block with a region attribute having a sudden change in the image, or searching an irregular image block with a certain visual significance consisting of adjacent pixels with similar texture, color, brightness and other characteristics in the image, and then extracting the characteristic points in the image block. Finally, matching the characteristic points in the black-and-white image with the corresponding points in the color image;
step three: the black-white-color image fusion problem is converted into an image coloring problem;
step four: recovering color information of the black and white image by a low-rank matrix filling method;
The binocular image enhancement problem of the different modes can be translated into an image rendering problem and can be expressed as:
（4）
In the formula (I), the compound is shown in the specification,representing a color image to be sought after,in which the representation is made of black-and-white imagesa matrix of pixel values of similar image blocks,AndPixel positions and pixel values respectively representing color information of corresponding points in the color image of the black-and-white image,representing a linear transformation of a color image into a black-and-white image,which represents the hadamard product, is,representing the Frobenius norm. To promotelow rank, can order
（7）
in the formula (I), the compound is shown in the specification,representing the nuclear norm. In this case, (4) is equivalent to
（8）
To effectively solve equation (6), it can be rewritten as:
（5）
with an augmented Lagrange multiplier of
（6）
in the formula (I), the compound is shown in the specification,Representing the traces of the matrix, which can be alternately updated by the alternative direction multiplier methodand until convergence (as shown in equation (9) and equation (10)), solving the problem of equation (5):
（9）
（10）。
as described above, the present invention converts the multispectral image enhancement problem to the low-rank matrix filling problem based on a black-and-white-color binocular camera. Aiming at the problems that time complexity is high and practical use is difficult due to accurate stereo matching, an image enhancement method based on a black-white-color camera is provided, black-white image feature points and a color image are matched, the color values of the matching points in the color image are regarded as the color values of the black-white image feature points, and the color of the black-white image is recovered through the pixel points with known color values by using a low-rank matrix filling method. The method can effectively improve the operation efficiency, improves the image sharpness, enhances the image details, inhibits the image noise and has very important significance in the field of image processing.
description of the drawings:
FIG. 1 is a schematic diagram of a color filter array;
FIG. 2 is a schematic diagram of a black and white camera sensor;
FIG. 3 is a schematic diagram of spectral sensitivities of a black-and-white camera and a color camera;
FIG. 4 is a schematic diagram of an algorithm;
FIG. 5 is an image taken by a black and white camera in this experiment;
FIG. 6 is an image taken by a color camera in this experiment;
FIG. 7 is feature point extraction;
fig. 8 is a corresponding relationship between a black-and-white image and a color image estimated in this experiment;
fig. 9 is a final restored color image.
the specific implementation mode is as follows:
referring to fig. 4, the present invention will be described in detail below with reference to the accompanying drawings, where the image enhancement method based on a monochrome-color camera according to the present embodiment includes the following steps:
the method comprises the following steps: a black-white-color binocular camera system is utilized to obtain a group of black-white-color images, the original black-white images and the original black-white images of the experiment are respectively shown in fig. 5 and fig. 6, and the image size is 3968 × 2976;
Step two: selecting proper characteristic points in the black-and-white image, and matching the characteristic points of the black-and-white image in the color image, wherein the method specifically comprises the following steps: first, feature points of a black-and-white image are extracted, and the feature points extracted in this experiment are shown in fig. 7. Then, matching the feature points in the black-and-white image with the matching points in the color image, wherein the matching result of the experiment is shown in fig. 8;
Step three: the black-white-color image fusion problem is converted into an image coloring problem;
Step four: and recovering the color information of the black and white image by a low-rank matrix filling method. And (3) respectively and alternately updating the new type (9) and the formula (10) by using an alternate direction multiplier method to restore the color of the black-white image:
（9）
（10）
In the formula (I), the compound is shown in the specification,representing a color image to be sought after,representing a matrix of pixel values of similar image blocks in a black and white image,And pixel positions and pixel values respectively representing color information of corresponding points in the color image of the black-and-white image,Representing a linear transformation of a color image into a black-and-white image,Which represents the hadamard product, is,represents the Frobenius norm,a trace representing a matrix;
the alternating direction multiplier method is a method for solving an optimization problem having the form:
（11）
The augmented Lagrange multiplier term is
（12）
the algorithm framework for solving this problem by the alternating direction multiplier method is as follows:
ADMM algorithm steps
Given、，AndandAn initial value of (d);
2 calculation of，，，；
judging whether the requirements are met, if so, executing the step IV; if not, executing the fifth step;
fourthly, calculating，，；
value is assigned
sixthly, calculate
seventhly, finishing the judgment of the step III
repeating the calculation steps from the second step to the third step until the calculation is completedStopping the operation and outputting the result;
to solve the equations (9) and (10), letIs composed of，Is composed ofIs ait can be solved by the framework of the alternative direction multiplier method, and the result of this experiment is shown in FIG. 9
as described above, the present invention converts the multispectral image enhancement problem to the low-rank matrix filling problem based on a black-and-white-color binocular camera. Aiming at the problems that time complexity is high and practical use is difficult due to accurate stereo matching, an image enhancement method based on a black-white-color camera is provided, black-white image feature points and a color image are matched, the color values of the matching points in the color image are regarded as the color values of the black-white image feature points, and the color of the black-white image is recovered through the pixel points with known color values by using a low-rank matrix filling method. The method not only avoids complex stereo matching and improves the operation efficiency, but also improves the image details, the image sharpness, the image signal to noise ratio and other aspects of the result image relative to the original image, and has very important significance in the field of image processing.
Claims (4)
1. An image enhancement method based on a black-and-white-color binocular camera is characterized in that binocular image enhancement problems of different modes are converted into a low-rank matrix filling problem, and can be expressed as:
（1）
In the formula (I), the compound is shown in the specification,representing a color image to be sought after,a luminance fidelity term is represented by a number of luminance fidelity terms,A color fidelity term is represented by a color fidelity term,a regularization term is represented as a function of,the representation is made of a black-and-white image,the color value of the corresponding point of the black-and-white image in the color image is represented, the brightness fidelity item is used for promoting the recovered image brightness to approach the pixel value of the black-and-white image, the color fidelity item is used for promoting the recovered image color to approach the color value of the corresponding point of the black-and-white image in the color image, and the brightness fidelity item and the color fidelity item used in the patent are respectively as follows:
（2）
（3）
in the formula (I), the compound is shown in the specification,indicating the pixel position of the corresponding point in the color image of the black-and-white image,representing a linear transformation of a color image into a black-and-white image,which represents the hadamard product, is,the norm is expressed as Frobenius norm when usedin this case, (1) is equivalent to:
（4）。
2. the black-and-white-color binocular camera-based image enhancement method of claim 1, wherein the problem of formula (4) is solved by an alternating direction multiplier method.
3. the image enhancement method based on the black-and-white-color binocular camera according to claim 1 or 2, comprising the steps of:
the method comprises the following steps: acquiring a group of black-white-color images by using a black-white-color binocular camera system;
step two: selecting proper characteristic points in the black-and-white image, and searching corresponding points of the characteristic points in the color image;
Step three: the black-white-color image fusion problem is converted into an image coloring problem;
step four: and recovering the color information of the black and white image by a low-rank matrix filling method.
4. the black-and-white-and-color binocular camera-based image enhancement method according to claim 1, 2 or 3, wherein the low rank matrix filling method in step four is to alternately update the problem shown in formula (1) by an alternating direction multiplier methodAndand (6) solving.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810591239.9A CN110580684A (en) | 2018-06-10 | 2018-06-10 | image enhancement method based on black-white-color binocular camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810591239.9A CN110580684A (en) | 2018-06-10 | 2018-06-10 | image enhancement method based on black-white-color binocular camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110580684A true CN110580684A (en) | 2019-12-17 |
Family
ID=68809938
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810591239.9A Pending CN110580684A (en) | 2018-06-10 | 2018-06-10 | image enhancement method based on black-white-color binocular camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110580684A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110987183A (en) * | 2019-12-27 | 2020-04-10 | 广州极飞科技有限公司 | Multispectral imaging system and method |
| CN111861911A (en) * | 2020-06-29 | 2020-10-30 | 湖南傲英创视信息科技有限公司 | Method and system for enhancing stereoscopic panoramic image based on guide camera |
| CN113129400A (en) * | 2021-03-17 | 2021-07-16 | 维沃移动通信有限公司 | Image processing method, image processing device, electronic equipment and readable storage medium |
| CN118138740A (en) * | 2024-03-11 | 2024-06-04 | 杭州非白三维科技有限公司 | Hand-held high-precision three-dimensional scanning array structure of four-eye camera, vision method and system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5825917A (en) * | 1994-09-30 | 1998-10-20 | Sanyo Electric Co., Ltd. | Region-based image processing method, image processing apparatus and image communication apparatus |
| EP0930777A1 (en) * | 1995-07-12 | 1999-07-21 | Sanyo Electric Co., Ltd. | Region-based image processing method, image processing apparatus and image communication apparatus |
| CN102567714A (en) * | 2011-12-14 | 2012-07-11 | 深圳市中控生物识别技术有限公司 | Method for correcting color image and black-and-white image based on double-camera face identification |
| CN103559687A (en) * | 2013-10-28 | 2014-02-05 | 上海理工大学 | Processing method for enabling black and white photographing system to recover color information |
| CN104978573A (en) * | 2015-07-06 | 2015-10-14 | 河海大学 | Non-negative matrix factorization method applied to hyperspectral image processing |
| CN106488107A (en) * | 2015-08-31 | 2017-03-08 | 宇龙计算机通信科技(深圳)有限公司 | A kind of image combining method based on dual camera and device |
| CN107292337A (en) * | 2017-06-13 | 2017-10-24 | 西北工业大学 | Ultralow order tensor data filling method |
-
2018
- 2018-06-10 CN CN201810591239.9A patent/CN110580684A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5825917A (en) * | 1994-09-30 | 1998-10-20 | Sanyo Electric Co., Ltd. | Region-based image processing method, image processing apparatus and image communication apparatus |
| EP0930777A1 (en) * | 1995-07-12 | 1999-07-21 | Sanyo Electric Co., Ltd. | Region-based image processing method, image processing apparatus and image communication apparatus |
| CN102567714A (en) * | 2011-12-14 | 2012-07-11 | 深圳市中控生物识别技术有限公司 | Method for correcting color image and black-and-white image based on double-camera face identification |
| CN103559687A (en) * | 2013-10-28 | 2014-02-05 | 上海理工大学 | Processing method for enabling black and white photographing system to recover color information |
| CN104978573A (en) * | 2015-07-06 | 2015-10-14 | 河海大学 | Non-negative matrix factorization method applied to hyperspectral image processing |
| CN106488107A (en) * | 2015-08-31 | 2017-03-08 | 宇龙计算机通信科技(深圳)有限公司 | A kind of image combining method based on dual camera and device |
| CN107292337A (en) * | 2017-06-13 | 2017-10-24 | 西北工业大学 | Ultralow order tensor data filling method |
Non-Patent Citations (1)
| Title |
|---|
| QUANMING YAO JAMES T. KWOK: "Colorization by Patch-Based Local Low-Rank Matrix Completion", 《PROCEEDINGS OF THE TWENTY-NINTH AAAI CONFERENCE ON ARTIFIVIAL INTELLIGENCE》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110987183A (en) * | 2019-12-27 | 2020-04-10 | 广州极飞科技有限公司 | Multispectral imaging system and method |
| CN111861911A (en) * | 2020-06-29 | 2020-10-30 | 湖南傲英创视信息科技有限公司 | Method and system for enhancing stereoscopic panoramic image based on guide camera |
| CN111861911B (en) * | 2020-06-29 | 2024-04-16 | 湖南傲英创视信息科技有限公司 | Stereoscopic panoramic image enhancement method and system based on guiding camera |
| CN113129400A (en) * | 2021-03-17 | 2021-07-16 | 维沃移动通信有限公司 | Image processing method, image processing device, electronic equipment and readable storage medium |
| CN118138740A (en) * | 2024-03-11 | 2024-06-04 | 杭州非白三维科技有限公司 | Hand-held high-precision three-dimensional scanning array structure of four-eye camera, vision method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111080724B (en) | Fusion method of infrared light and visible light | |
| CN110660088B (en) | Image processing method and device | |
| US11830222B2 (en) | Bi-level optimization-based infrared and visible light fusion method | |
| WO2021120406A1 (en) | Infrared and visible light fusion method based on saliency map enhancement | |
| CN111741281B (en) | Image processing method, terminal and storage medium | |
| CN104683767B (en) | Penetrating Fog image generating method and device | |
| CN111510691B (en) | Color interpolation method and device, equipment and storage medium | |
| EP4070268A1 (en) | Deep residual network for color filter array image denoising | |
| CN109360235A (en) | A kind of interacting depth estimation method based on light field data | |
| CN113454680A (en) | Image processor | |
| CN108055452A (en) | Image processing method, device and equipment | |
| CN108024054A (en) | Image processing method, device and equipment | |
| JP6290392B2 (en) | Conversion of images from dual-band sensors into visible color images | |
| RU2690757C1 (en) | System for synthesis of intermediate types of light field and method of its operation | |
| CN110580684A (en) | image enhancement method based on black-white-color binocular camera | |
| CN114868384B (en) | Apparatus and method for image processing | |
| CN107220955A (en) | A kind of brightness of image equalization methods based on overlapping region characteristic point pair | |
| KR20180118432A (en) | Image Processing Apparatus and Method for Improving Sensitivity | |
| CN114996814A (en) | Furniture design system based on deep learning and three-dimensional reconstruction | |
| CN116703752A (en) | Image defogging method and device of near infrared fused transducer structure | |
| CN114820581A (en) | Axisymmetric optical imaging parallel simulation method and device | |
| JP5718138B2 (en) | Image signal processing apparatus and program | |
| CN102223545B (en) | Rapid multi-view video color correction method | |
| Sheng et al. | Guided colorization using mono-color image pairs | |
| CN110910457B (en) | Multispectral three-dimensional camera external parameter calculation method based on angular point characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20191217 |
|
| WD01 | Invention patent application deemed withdrawn after publication |