CN115334294A - Video noise reduction method of local self-adaptive strength - Google Patents
Video noise reduction method of local self-adaptive strength Download PDFInfo
- Publication number
- CN115334294A CN115334294A CN202210775955.9A CN202210775955A CN115334294A CN 115334294 A CN115334294 A CN 115334294A CN 202210775955 A CN202210775955 A CN 202210775955A CN 115334294 A CN115334294 A CN 115334294A
- Authority
- CN
- China
- Prior art keywords
- local
- bandwidth
- curve
- noise reduction
- brightness
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Picture Signal Circuits (AREA)
Abstract
The invention belongs to the technical field of image signal processing, and particularly relates to a local adaptive strength video denoising method. The method comprises the following steps: generating a local gain map; generating a local bandwidth by matching with the local brightness and the BW0 and BW1 bandwidth curves; taking the local bandwidth as a 3dB bandwidth control point, and obtaining a final weight value through a similarity-to-weight curve; and finally, performing weighting, limiting and original image fusion to obtain a final noise reduction result. The method can accurately obtain the local gain map, and obtain the local bandwidth by matching with the brightness noise curve, thereby fundamentally simulating noise models influenced by digital gain at all levels, and adapting to different noise reduction strategies for each pixel of the same frame to achieve more accurate and better local noise reduction effect.
Description
Technical Field
The invention belongs to the technical field of image signal processing, and particularly relates to a video noise reduction method based on local adaptive strength.
Background
The raw image sensor sends bare data, which contains various noises, and the noise can be divided into random noise and fixed noise, the random noise is irrelevant to period and position, and the fixed noise appears at a fixed position of an image because the characteristics of pixels are inconsistent. These noises deteriorate the image quality and also determine the sensitivity of the image sensor. Corresponding noise reduction methods are required for different noise types, and the main technologies of video noise reduction include space domain noise reduction, time domain noise reduction, transform domain noise reduction and the like.
The common video noise reduction is carried out in a Bayer domain or a YUV domain, and the Bayer domain generally adopts spatial noise reduction. Bayer domain noise reduction is typically located on the pipeline in the bayer linear domain, such as before Tonemapping. The model which furthest retains the original noise is subjected to noise reduction in the Bayer domain, so that a better noise reduction effect is achieved, and meanwhile, the cost for noise reduction in the Bayer domain is higher, and the technical difficulty is higher. However, the bayer linear domain is not completely linear, because the noise distribution is damaged to different degrees due to the existence of various digital gains, and the digital gains are changed according to different regions and different scenes. Common bayer domain noise reduction algorithms such as bilateral filtering based and traditional non-local mean algorithm based have no way to solve the above problems.
The invention provides a video noise reduction method with local self-adaptive strength, which replaces the traditional fitting mode, fundamentally simulates various noise models damaged by digital gain, and is adapted to different pixels in the same frame to use different noise reduction strategies. The factors influencing the noise form are used as input, local noise reduction bandwidth is output, the bandwidth and a similarity-to-weight curve determine the weight of the current pixel participating in noise reduction, and finally the local self-adaptive noise reduction degree is realized on the noise reduction effect of the current pixel.
Disclosure of Invention
The invention aims to provide a video denoising method with local self-adaptive strength aiming at the problem of noise model damage in Bayer domain linear denoising.
The video noise reduction method of the local adaptive strength provided by the invention adopts the local adaptive strength method to replace the traditional fitting mode, fundamentally simulates various noise models damaged by digital gain, and adapts to different noise reduction strategies for different pixels in the same frame. And taking factors influencing noise forms as input, outputting local noise reduction bandwidth, determining the weight of the current pixel participating in noise reduction by matching the bandwidth with a similarity-to-weight curve, and finally reflecting the noise reduction effect of the current pixel to realize local self-adaptive noise reduction.
The invention provides a video denoising method with local adaptive strength, which comprises the following specific steps:
(1) Generating a local gain map;
(2) Generating a local bandwidth by matching with the local brightness and BW0, BW1 bandwidth curves;
(3) Taking the local bandwidth as a 3dB bandwidth control point, and converting the local bandwidth into a weight curve through the similarity to obtain a final weight value;
(4) And finally, performing weighting, limiting and original image fusion to obtain a final noise reduction result.
The step (1) of generating the local gain map comprises the following specific processes: various digital gain signals such as digital gain of automatic exposure control, automatic white balance channel gain, lens shadow correction channel gain, channel gain (including motion area and overexposure area) used in high dynamic range fusion and the like are multiplied point by point and converged together, the converging process needs functions such as storage, synchronization and the like, all pixels are completely aligned, and finally a local gain map is generated.
And (3) generating a local bandwidth by matching the local brightness and the bandwidth curves of BW0 and BW1, wherein the specific process is as follows:
(2.1) extracting pixels of the same channel from the original Bayer image, taking the current pixel as a 7x7 brightness window of the center, and obtaining the brightness of the center pixel through a low-pass filter, namely: local brightness;
(2.2) sending the local gain map obtained in the step (1) to a local bandwidth 0 curve (BW 0) module (figure 3), dividing local brightness by the local gain map, and then sending to a local bandwidth 1 curve (BW 1) module (figure 4); the Local bandwidth 0 curve (BW 0) inputs Local gain and outputs Local _ BW0, wherein the Local bandwidth 0 curve adopts 16-segment fitting curve (figure 3), the curve control point coefficient is calibrated according to different image sensors (CMOS sensors), and different gains correspond to different noise reduction bandwidths. The Local bandwidth 1 curve (BW 1) inputs the original brightness of the pixel (the original brightness is obtained by dividing the Local brightness by the Local gain), and outputs Local _ BW1, wherein the Local bandwidth 1 curve adopts a 16-segment fitting curve (fig. 4), the curve control point coefficient is calibrated according to different image sensors (cmos sensors), and different photosensitive brightness corresponds to different noise reduction bandwidths. And finally multiplying the local bandwidth 0 by the local bandwidth 1 to obtain the final local bandwidth.
In the step (3), the smaller the similarity value is, the larger the corresponding weight should be, and in the step (3), the local bandwidth is used as a 3dB bandwidth control point, and a final weight value is obtained through a similarity-to-weight curve (fig. 5); the specific process is as follows: taking a current pixel as a center, recording a similarity table of each point and a central point in a 7x7 window as S [7,7], recording the similarity table as 49 points, converting the similarity of the 49 points into corresponding 49 weight values, and recording the weight values as a weight table W [7,7]; specifically, a low-pass filter described in fig. 5 is used for fitting, and the 3dB bandwidth of the low-pass filter is controlled by the local bandwidth obtained in step (2), and is divided into 9 orders. Therefore, the noise reduction bandwidth of each pixel is realized, and each pixel obtains the most appropriate noise reduction strength according to the self condition. Similarly, based on the similarity-to-weight curve, the corresponding S [7,7]49 points can be converted into W [7,7] weight table, as shown in FIG. 6. Having thus obtained the weight of each 7x7 neighboring pixel required for noise reduction and weighting, the noise reduction process for the current pixel can be started.
And (4) performing weighting, limiting and original graph fusion to obtain a final noise reduction result, wherein the specific process is as follows: the noise reduction process of the 7x7 weight table W [7,7] generated in step (3) is relatively simple, and the final noise reduction output result can be obtained by only performing weighting, limiting and fusion with the original graph, as shown in fig. 7. The specific formula is as follows:
fusion _ o = NR _ o = alpha + raw [24] (1-alpha), where alpha = [ 0-1 ];
wherein, NR _ o is the noise-reduced result of the current pixel, and Fusion _ o is the output after the noise-reduced result of the current pixel and the pixel before the original noise reduction are fused; clip represents a truncate operation, where less than 0 is limited to 0 and greater than 4095 is limited to 4095; raw [24] represents 7 × 7 original bare data centered on the current pixel, and W [ i ] represents the weight corresponding to 7 × 7 pixels centered on the current pixel, i.e., W [7,7].
The specific benefit effects of the invention are: the method can accurately obtain a local gain map, and obtain local bandwidth by matching with a brightness noise curve, fundamentally simulates noise models influenced by digital gain at all levels, and adapts to different noise reduction strategies for each pixel of the same frame, thereby achieving more accurate and better local noise reduction effect.
Drawings
Fig. 1 is a flow chart of local bandwidth generation of local adaptive noise reduction.
Fig. 2 is local gain map generation.
Fig. 3 is a local bandwidth 0 curve.
Fig. 4 is a local bandwidth 1 curve.
Fig. 5 is a similarity to weight curve.
Fig. 6 is a 7x7 weight table generation.
FIG. 7 is a denoising fusion process.
FIG. 8 illustrates the location of the method of the present invention in the video denoising reference flow.
Detailed Description
The local adaptive strength noise reduction method of the present invention is embedded in a specific noise reduction algorithm, such as a local gain and weight calculation part in a video noise reduction RawNR (bare figure noise reduction), as shown in fig. 8. And inputting the naked video image and outputting the fused video with good noise reduction. The specific embodiment of the invention is as follows:
fig. 1 is a local bandwidth generation diagram, which is completed in a local gain module by first generating a local gain map, where the local gain is obtained by converging various digital gain signals such as digital gain for automatic exposure control, automatic white balance channel gain, lens shading correction channel gain, and channel gain used in high dynamic range fusion, and the convergence process requires line storage, synchronization, and other functions to completely align each pixel, and finally generating the local gain map. Meanwhile, the 7x7 brightness window sent by the channel window can obtain the local brightness of the center through a low-pass filter, the local gain map is sent to the local bandwidth 0 curve module, and the local brightness is divided by the local gain map and then sent to the local bandwidth 1 curve module. The local bandwidth 0 curve is calibrated according to different sensors by sending out a local bandwidth 0,16 section fitting curve according to input local gain, and different gains correspond to different noise reduction bandwidths. The local bandwidth 1 curve is sent out a local bandwidth 1,16 segment fitting curve according to the original brightness after the input local brightness is divided by the local gain, calibration is completed according to different sensors, and different photosensitive brightness corresponds to different noise reduction bandwidths. And finally multiplying the local bandwidth 0 by the local bandwidth 1 to obtain the final local bandwidth.
Fig. 2 is a detailed process of local gain map generation, since a general pipeline design has modules such as WBG (white balance gain), LSC (lens shading correction), HDRFusion (high dynamic range) and the like before RawNR, where WBG includes global automatic white balance gain and automatic exposure digital gain, LSC includes local lens shading correction luminance shading and color shading channel gain, HDRFusion includes different gains of selecting different regions in the fusion process using long/medium/short exposure, and the like, and these different gains will eventually change the noise shape of each pixel. The local gain map of the upper graph is a complete local gain map, which is formed by fusing all the related digital gains together, and is accurate to each pixel.
Fig. 3 is a local bandwidth 0 curve, which is obtained by fitting 16 segments of dotted lines with local gains as input, and different local gains correspond to different local bandwidths 0 (intensity of noise), and are obtained by calibrating a specifically used image sensor.
Fig. 4 is a local bandwidth 1 curve, which is obtained by fitting 16 segments of dotted lines with the local brightness divided by the original local brightness of the local gain as input, and different local original brightness corresponds to different local bandwidths 1 (intensity of noise), and is obtained by calibrating the image sensor used specifically.
The outputs of the two curves are multiplied to obtain the final local bandwidth, which is used as a parameter to be sent to the similarity-to-weight curve, as shown in fig. 5.
The smaller the S value, the greater the corresponding weight, and here the low pass filter described in fig. 5 is used to fit, divided into 9 orders, with the 3dB bandwidth of the low pass filter being governed by the local bandwidth obtained above. Therefore, the noise reduction bandwidth of each pixel is realized, and each pixel obtains the most appropriate noise reduction strength according to the self condition.
According to the similarity-to-weight curve, the corresponding S7 points 49 points can be converted into a W7 weight table, as shown in FIG. 6. Having thus obtained the weight of each 7x7 neighboring pixel required for noise reduction and weighting, the noise reduction process for the current pixel can be started.
The denoising process with the weight of 7x7 is relatively simple, and the final denoising output result can be obtained only by weighting, limiting and fusing with the original graph.
Claims (3)
1. A video denoising method with local adaptive strength is characterized by comprising the following specific steps:
(1) Generating a local gain map;
(2) Generating a local bandwidth by matching with the local brightness and BW0, BW1 bandwidth curves;
(3) Taking the local bandwidth as a 3dB bandwidth control point, and converting the similarity into a weight curve to obtain a weight value;
(4) Finally, performing weighting, limiting and original graph fusion to obtain a final noise reduction result;
the step (1) of generating the local gain map comprises the following specific processes: multiplying and converging various digital gain signals of digital gain of automatic exposure control, automatic white balance channel gain, lens shadow correction channel gain and channel gain used in high dynamic range fusion point by point, storing and synchronizing the converging process, completely aligning each pixel, and finally generating a local gain map;
in the step (2), the BW0 bandwidth curve is also called a local bandwidth 0 curve, and the BW1 bandwidth curve is also called a local bandwidth 1 curve; the local bandwidth is generated by matching the local brightness and the BW0, BW1 bandwidth curves, and the specific flow is as follows:
(2.1) extracting pixels of the same channel from the original Bayer image, and obtaining the brightness of a central pixel through a low-pass filter by using a 7x7 brightness window taking the current pixel as the center, namely: local brightness;
(2.2) sending the local gain map obtained in the step (1) to a local bandwidth 0 curve module, dividing local brightness by the local gain map, and then sending to a local bandwidth 1 curve module;
the Local bandwidth 0 curve inputs Local gain and outputs Local _ BW0; here, the local bandwidth 0 curve adopts 16 segments of fitting curves, the curve control point coefficients are calibrated according to different image sensors, and different gains correspond to different noise reduction bandwidths;
inputting pixel original brightness by a Local bandwidth 1 curve, wherein the original brightness is obtained by dividing the Local brightness by Local gain, and outputting Local _ BW1, the Local bandwidth 1 curve adopts a 16-segment fitting curve, the curve control point coefficient is calibrated according to different image sensors, and different photosensitive brightness corresponds to different noise reduction bandwidths;
and finally multiplying the local bandwidth 0 by the local bandwidth 1 to obtain the final local bandwidth.
2. The method for video denoising with local adaptive power according to claim 1, wherein in step (3), the smaller the similarity value is, the larger the corresponding weight should be, and in step (3), the local bandwidth is taken as a 3dB bandwidth control point, and the final weight value is obtained by a similarity-to-weight curve; the specific process is as follows: taking a current pixel as a center, recording a similarity table of each point and a central point in a 7x7 window as S [7,7], converting the similarity table into a corresponding weight table with 49 point similarities, and recording the weight table as W [7,7]; specifically, a low-pass filter is adopted for fitting and is divided into 9 orders, the 3dB bandwidth of the low-pass filter is controlled by the local bandwidth obtained in the step (2), so that the noise reduction bandwidth of each pixel is realized, and each pixel obtains the most appropriate noise reduction strength according to the self condition; in the same way, according to the similarity-to-weight curve, the corresponding S [7,7]49 points are converted into a W [7,7] weight table, and the weight of each 7x7 adjacent pixel required by noise reduction and weight calculation is obtained.
3. The method for video denoising with local adaptive power according to claim 2, wherein the weighting, limiting and original map fusion in step (4) are performed to obtain the final denoising result, and the specific formula is as follows:
fusion _ o = NR _ o = alpha + raw [24] (1-alpha), where alpha = [0 to 1];
wherein, NR _ o is the noise-reduced result of the current pixel, and Fusion _ o is the output of the fused noise-reduced result of the current pixel and the pixel before the original noise reduction; clip represents a truncate operation, where less than 0 is limited to 0 and greater than 4095 is limited to 4095; raw [24] represents 7 × 7 original bare data centered on the current pixel, and W [ i ] represents the weight corresponding to 7 × 7 pixels centered on the current pixel, i.e., W [7,7].
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210775955.9A CN115334294B (en) | 2022-07-02 | 2022-07-02 | Video noise reduction method of local self-adaptive force |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210775955.9A CN115334294B (en) | 2022-07-02 | 2022-07-02 | Video noise reduction method of local self-adaptive force |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115334294A true CN115334294A (en) | 2022-11-11 |
| CN115334294B CN115334294B (en) | 2023-06-20 |
Family
ID=83916985
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210775955.9A Active CN115334294B (en) | 2022-07-02 | 2022-07-02 | Video noise reduction method of local self-adaptive force |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115334294B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115003A (en) * | 2023-02-15 | 2023-11-24 | 荣耀终端有限公司 | Method and device for removing noise |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150023594A1 (en) * | 2013-07-16 | 2015-01-22 | Texas Instruments Incorporated | Transforming Wide Dynamic Range Images to Reduced Dynamic Range Images |
| CN106339989A (en) * | 2016-08-13 | 2017-01-18 | 浙江莱达信息技术有限公司 | Medical X-ray image adaptive noise reduction method |
| CN112819721A (en) * | 2021-02-04 | 2021-05-18 | 湖南兴芯微电子科技有限公司 | Method and system for reducing noise of image color noise |
-
2022
- 2022-07-02 CN CN202210775955.9A patent/CN115334294B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150023594A1 (en) * | 2013-07-16 | 2015-01-22 | Texas Instruments Incorporated | Transforming Wide Dynamic Range Images to Reduced Dynamic Range Images |
| CN106339989A (en) * | 2016-08-13 | 2017-01-18 | 浙江莱达信息技术有限公司 | Medical X-ray image adaptive noise reduction method |
| CN112819721A (en) * | 2021-02-04 | 2021-05-18 | 湖南兴芯微电子科技有限公司 | Method and system for reducing noise of image color noise |
Non-Patent Citations (1)
| Title |
|---|
| 崔建伟;谷源涛;唐昆;: "采用背景提取和自适应滤波的视频降噪算法", 电视技术, no. 2 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115003A (en) * | 2023-02-15 | 2023-11-24 | 荣耀终端有限公司 | Method and device for removing noise |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115334294B (en) | 2023-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11849224B2 (en) | Global tone mapping | |
| CN110378859B (en) | Novel high dynamic range image generation method | |
| US9445067B2 (en) | Imaging device and image signal processor with color noise correction | |
| CN109785240B (en) | Low-illumination image enhancement method and device and image processing equipment | |
| CN110930301B (en) | Image processing method, device, storage medium and electronic equipment | |
| CN110113510B (en) | Real-time video image enhancement method and high-speed camera system | |
| WO2012042771A1 (en) | Image processor, image processing method, and integrated circuit | |
| US20090002530A1 (en) | Apparatus and method for processing images | |
| JP4810473B2 (en) | Image processing apparatus and image processing program | |
| CN109816608A (en) | A kind of low-light (level) image adaptive brightness enhancement based on noise suppressed | |
| KR101639664B1 (en) | Photographing apparatus and photographing method | |
| CN107862672B (en) | Image defogging method and device | |
| CN111724317A (en) | Method for constructing Raw domain video denoising supervision data set | |
| WO2019104047A1 (en) | Global tone mapping | |
| CN114862698A (en) | Method and device for correcting real overexposure image based on channel guidance | |
| CN115334294A (en) | Video noise reduction method of local self-adaptive strength | |
| CN115984535A (en) | Low-illumination image judgment and image enhancement method for drilling operation site | |
| CN115526806A (en) | Automatic black-light image color correction method based on artificial intelligence | |
| WO2023005115A1 (en) | Image processing method, image processing apparatus, electronic device, and readable storage medium | |
| CN112488968B (en) | Image enhancement method for hierarchical histogram equalization fusion | |
| CN116843584B (en) | Image data optimization enhancement method | |
| CN113362253A (en) | Image shadow correction method, system and device | |
| CN115526811B (en) | Adaptive vision SLAM method suitable for variable illumination environment | |
| KR20120122574A (en) | Apparatus and mdthod for processing image in a digital camera | |
| EP4167134A1 (en) | System and method for maximizing inference accuracy using recaptured datasets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |