CN111429363A - Video noise reduction method based on video coding - Google Patents
Video noise reduction method based on video coding Download PDFInfo
- Publication number
- CN111429363A CN111429363A CN202010113101.5A CN202010113101A CN111429363A CN 111429363 A CN111429363 A CN 111429363A CN 202010113101 A CN202010113101 A CN 202010113101A CN 111429363 A CN111429363 A CN 111429363A
- Authority
- CN
- China
- Prior art keywords
- video
- image block
- noise reduction
- coding
- similar image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 230000004927 fusion Effects 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 5
- 239000013598 vector Substances 0.000 claims description 23
- 239000003086 colorant Substances 0.000 claims description 4
- 229910002056 binary alloy Inorganic materials 0.000 claims description 3
- 238000011524 similarity measure Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 9
- 230000005540 biological transmission Effects 0.000 abstract description 5
- 238000011946 reduction process Methods 0.000 abstract description 3
- 230000001133 acceleration Effects 0.000 abstract 1
- 238000005549 size reduction Methods 0.000 abstract 1
- 238000001514 detection method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
- H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a video noise reduction method based on video coding, which comprises the following steps: s1, carrying out video coding processing on the video to be denoised, and constructing a coding path corresponding to a denoising image block of the current image frame; s2, according to the coding path, giving different mixing weights to each similar image block; s3, according to the coding path, carrying out consistency check on the similar image blocks and the noise reduction image blocks to obtain corresponding image block masks; s4, according to the mixed weight and the image block mask, performing image block fusion to complete one-frame video noise reduction; s5, completing the video noise reduction of the continuous frames according to the steps S1-S5. The method of the invention integrates the advantages of video coding, video file size reduction, video transmission acceleration, video noise reduction and video noise interference reduction, combines the two technologies, and realizes a fast and efficient video noise reduction process.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a video noise reduction method based on video coding.
Background
Video coding is an important technology widely applied to the field of video compression and video transmission, and the video coding technology reduces the size of a video file to a great extent by reducing the redundancy of video in space and time, so that the possibility of fast video transmission is provided.
Video noise reduction is to reduce noise interference in an image by using information on time and space, so as to generate a clear image. In daily life, due to the existence of dark light, insufficient exposure time and other reasons, the shot video is often subjected to serious noise interference. Therefore, video denoising techniques are required for post-processing of video.
Video coding and video noise reduction technologies are widely applied in respective fields, but currently, a corresponding technology is still lacked to combine the advantages of the two technologies, so as to efficiently realize video noise reduction.
Disclosure of Invention
In order to overcome the defects in the prior art, the video noise reduction method based on video coding combines the video coding and video noise reduction technologies, and achieves noise reduction rapidly and efficiently.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that: a video noise reduction method based on video coding comprises the following steps:
s1, carrying out video coding processing on the video to be denoised, and constructing a coding path corresponding to a denoising image block of the current image frame;
recording the motion vector of the similar image block corresponding to each noise-reduced image block in the coding path;
s2, according to the coding path, giving different mixing weights to each similar image block;
s3, according to the coding path, carrying out consistency check on the similar image blocks and the noise reduction image blocks to obtain corresponding image block masks;
s4, according to the mixed weight and the image block mask, performing image block fusion to complete one-frame video noise reduction;
s5, completing the video noise reduction of the continuous frames according to the steps S1-S4.
Further, in the step S1, the step S1 specifically includes:
s11, automatically dividing different areas of the video image into image blocks with different sizes according to the richness of the texture information;
s12, matching corresponding similar image blocks for each noise-reduced image block by calculating Euclidean distances between pixels of each image block;
s13, storing the motion vector and residual error information of each group of similar image blocks as a compressed code stream in a binary system mode, and realizing video coding;
s14, uniformly dividing all similar image blocks into image blocks with the size smaller than the original size of the similar image blocks, and enabling each divided small image block to have the same motion vector with the original similar image block;
and S15, linking the motion vectors corresponding to all the divided similar image blocks, and representing the motion vectors by different colors to form corresponding coding paths.
Further, the image block sizes in the step S11 include 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4;
the size of the small image block divided in step S14 is 4 × 4.
Further, in step S2, the index SSIM is used as a similarity measure to give a mixing weight to the image block corresponding to the encoding path;
mixing weight ω of each of the image blocksbComprises the following steps:
further, in step S3, specifically, the method includes:
sequentially judging whether the difference D between the pixel value of each similar image block and the pixel value of the noise-reduced image block is greater than a set threshold value or not;
if yes, the similar image block is invalid and fails consistency check;
if not, the similar image block is valid and passes consistency check.
Further, when the consistency check is performed in step S3, a threshold value is set to 20;
the tile mask is 12 × 12 in size and is represented as:
in the formula, K ═ 0 indicates that the current similar image block is invalid;
further, in step S4, the formula for image block fusion is:
in the formula, BfinalA frame of video image after noise reduction;
n is the number of similar image blocks for noise reduction;
Kia mask for the ith similar image block;
Biis the i-th similar image block for fusion.
The invention has the beneficial effects that:
the video noise reduction method based on video coding provided by the invention realizes the fast noise reduction of the video by introducing the video coding information, integrates the advantages of reducing the size of a video file by video coding, accelerating the video transmission and reducing the video noise interference by video noise reduction, combines the two technologies, and realizes the fast and efficient video noise reduction process.
Drawings
Fig. 1 is a flowchart of a video denoising method based on video coding according to the present invention.
FIG. 2 is a diagram illustrating image block partitioning and encoding paths according to the present invention.
FIG. 3 is a schematic diagram of the consistency detection provided by the present invention.
FIG. 4 is a schematic diagram illustrating comparison before and after noise reduction of a frame of image according to the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
As shown in fig. 1, a video noise reduction method based on video coding includes the following steps:
s1, carrying out video coding processing on the video to be denoised, and constructing a coding path corresponding to a denoising image block of the current image frame;
recording the motion vector of the similar image block corresponding to each noise-reduced image block in the coding path;
s2, according to the coding path, giving different mixing weights to each similar image block;
s3, according to the coding path, carrying out consistency check on the similar image blocks and the noise reduction image blocks to obtain corresponding image block masks;
s4, according to the mixed weight and the image block mask, performing image block fusion to complete one-frame video noise reduction;
s5, completing the video noise reduction of the continuous frames according to the steps S1-S4.
In the video encoding of step S1, the video frame may be divided into I frame, B frame, and P frame; wherein the I frame is a start frame; b frames can be encoded with reference to previous and subsequent frames; p frames can only be coded by referring to previous frames, and in the invention, H.264 coding is adopted, the first frame of the video is set as an I frame, and the use of P frames and B frames is allowed. Therefore, the step S1 is specifically:
s11, automatically dividing different areas of the video image into image blocks with different sizes according to the richness of the texture information, wherein the image blocks specifically comprise 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8 and 4 × 4;
s12, matching corresponding similar image blocks for each noise-reduced image block by calculating Euclidean distances between pixels of each image block;
s13, storing the motion vector and residual error information of each group of similar image blocks as a compressed code stream in a binary system mode, and realizing video coding;
in the above steps S12 and S13, in order to achieve the purpose of compression, other image blocks similar to the current image block need to be found by an exhaustive method within a frame and between frames, that is, the euclidean distance between pixels of the image block is calculated, and finally, the similar image block and corresponding motion vector information (motion vector, representing the displacement between two similar image blocks) can be matched for each image block, and the residual information (the residual between two image blocks) is stored as a compressed code stream in a binary form;
s14, uniformly dividing all similar image blocks into image blocks with the size smaller than the original size of the similar image blocks, and enabling each divided small image block to have the same motion vector with the original similar image block;
specifically, the divided small image blocks have a size of 4 × 4, enjoy a motion vector identical to that of the large image blocks, for example, an image block of 8 × 8 may be divided into 4 small image blocks of 4 × 4, which enjoy a motion vector of an image block of 8 × 8;
and S15, linking the motion vectors corresponding to all the divided similar image blocks, and representing the motion vectors by different colors to form corresponding coding paths.
As shown in fig. 2, each frame of the video is subdivided into uniform tiles, and each tile has other similar tiles, which are related by motion vectors (motion vectors). Therefore, starting from each image block needing noise reduction, a series of similar image blocks can be found through the guidance of motion vectors, and the motion vectors between the similar image blocks are linked to form a coding path (coding project); different colors represent different encoding paths.
In order to make the fusion more robust and have a larger coverage area, we extract the 12 × 12 image block for fusion, i.e. combine a plurality of small 4 × 4 image blocks into a large image block.
Because the similarity degrees between different image blocks and the image block to be denoised are different, different weights need to be given to the image blocks during fusion, and in the step S2, the index SSIM is used as a similarity measurement standard to give a mixed weight to the image block corresponding to the coding path;
mixing weight omega for each image blockbComprises the following steps:
influences brought by processing moving objects, such as ghost (left in fig. 3) and the like; the present invention proposes to reduce the influence by using a consistency check (consistency check), specifically setting a pixel-by-pixel comparison, so that step S3 specifically includes:
sequentially judging whether the difference D between the pixel value of each similar image block and the pixel value of the noise-reduced image block is greater than a set threshold value or not;
if yes, the similar image block is invalid and fails consistency check;
if not, the similar image block is valid and passes consistency check.
In performing the consistency check, a threshold of 20 is set, the size of the image patch mask is 12 × 12, and it is expressed as:
in the formula, K ═ 0 indicates that the current similar image block is invalid;
k1 is valid for the current similar image block.
In S4, the formula for image block fusion is:
in the formula, BfinalA frame of video image after noise reduction;
n is the number of similar image blocks for noise reduction;
Kia mask for the ith similar image block;
Biis the i-th similar image block for fusion.
In an embodiment of the present invention, a noise reduction visual effect map of the video noise reduction method of the present invention is provided, as shown in fig. 3((a) no consistent detection result, (b) consistent detection result) and fig. 4((a) before noise reduction, (b) after noise reduction), compared with the classical video noise reduction algorithm (VBM3D, VBM4D), the present invention can obtain comparable results in a shorter time (about 0.1 s).
The invention has the beneficial effects that:
the video noise reduction method based on video coding provided by the invention realizes the fast noise reduction of the video by introducing the video coding information, integrates the advantages of reducing the size of a video file by video coding, accelerating the video transmission and reducing the video noise interference by video noise reduction, combines the two technologies, and realizes the fast and efficient video noise reduction process.
Claims (7)
1. A video noise reduction method based on video coding is characterized by comprising the following steps:
s1, carrying out video coding processing on the video to be denoised, and constructing a coding path corresponding to a denoising image block of the current image frame;
recording the motion vector of the similar image block corresponding to each noise-reduced image block in the coding path;
s2, according to the coding path, giving different mixing weights to each similar image block;
s3, according to the coding path, carrying out consistency check on the similar image blocks and the noise reduction image blocks to obtain corresponding image block masks;
s4, according to the mixed weight and the image block mask, performing image block fusion to complete one-frame video noise reduction;
s5, completing the video noise reduction of the continuous frames according to the steps S1-S4.
2. The method for video denoising based on video coding according to claim 1, wherein in step S1, the step S1 is specifically:
s11, automatically dividing different areas of the video image into image blocks with different sizes according to the richness of the texture information;
s12, matching corresponding similar image blocks for each noise-reduced image block by calculating Euclidean distances between pixels of each image block;
s13, storing the motion vector and residual error information of each group of similar image blocks as a compressed code stream in a binary system mode, and realizing video coding;
s14, uniformly dividing all similar image blocks into image blocks with the size smaller than the original size of the similar image blocks, and enabling each divided small image block to have the same motion vector with the original similar image block;
and S15, linking the motion vectors corresponding to all the divided similar image blocks, and representing the motion vectors by different colors to form corresponding coding paths.
3. The video coding-based video denoising method of claim 2, wherein the tile sizes in step S11 include 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4;
the size of the small image block divided in step S14 is 4 × 4.
4. The video denoising method based on video coding of claim 1, wherein in step S2, an index SSIM is used as a similarity measure to give a mixing weight to the image block corresponding to the coding path;
mixing weight ω of each of the image blocksbComprises the following steps:
5. the method for video denoising based on video coding according to claim 1, wherein the step S3 specifically comprises:
sequentially judging whether the difference D between the pixel value of each similar image block and the pixel value of the noise-reduced image block is greater than a set threshold value or not;
if yes, the similar image block is invalid and fails consistency check;
if not, the similar image block is valid and passes consistency check.
6. The method for reducing noise in video according to claim 5, wherein when the consistency check is performed in step S3, the threshold is set to 20;
the tile mask is 12 × 12 in size and is represented as:
in the formula, K ═ 0 indicates that the current similar image block is invalid;
k1 is valid for the current similar image block.
7. The method for reducing noise in video according to claim 6, wherein in step S4, the formula for performing image block fusion is:
in the formula, BfinalA frame of video image after noise reduction;
n is the number of similar image blocks for noise reduction;
Kia mask for the ith similar image block;
Biis the i-th similar image block for fusion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010113101.5A CN111429363A (en) | 2020-02-24 | 2020-02-24 | Video noise reduction method based on video coding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010113101.5A CN111429363A (en) | 2020-02-24 | 2020-02-24 | Video noise reduction method based on video coding |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111429363A true CN111429363A (en) | 2020-07-17 |
Family
ID=71551561
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010113101.5A Pending CN111429363A (en) | 2020-02-24 | 2020-02-24 | Video noise reduction method based on video coding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111429363A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113139990A (en) * | 2021-05-08 | 2021-07-20 | 电子科技大学 | Depth grid stream robust image alignment method based on content perception |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101742288A (en) * | 2008-11-11 | 2010-06-16 | 北京中星微电子有限公司 | Video noise reduction encoding method and video noise reduction encoding device |
-
2020
- 2020-02-24 CN CN202010113101.5A patent/CN111429363A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101742288A (en) * | 2008-11-11 | 2010-06-16 | 北京中星微电子有限公司 | Video noise reduction encoding method and video noise reduction encoding device |
Non-Patent Citations (1)
| Title |
|---|
| ZHIHANG REN等: "CODING TRAJECTORY:ENABLE VIDEO CODING FOR VIDEO DENOISING", 《2018 25TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113139990A (en) * | 2021-05-08 | 2021-07-20 | 电子科技大学 | Depth grid stream robust image alignment method based on content perception |
| CN113139990B (en) * | 2021-05-08 | 2022-03-15 | 电子科技大学 | Depth grid stream robust image alignment method based on content perception |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111798400B (en) | Non-reference low-illumination image enhancement method and system based on generation countermeasure network | |
| CN111612708B (en) | Image restoration method based on countermeasure generation network | |
| CN108495135B (en) | Quick coding method for screen content video coding | |
| CN111432210B (en) | Point cloud attribute compression method based on filling | |
| CN107633482A (en) | A kind of super resolution ratio reconstruction method based on sequence image | |
| CN111355956A (en) | Rate distortion optimization fast decision making system and method based on deep learning in HEVC intra-frame coding | |
| CN111008608B (en) | Night vehicle detection method based on deep learning | |
| CN115297288B (en) | Monitoring data storage method for driving simulator | |
| EP0985318B1 (en) | System for extracting coding parameters from video data | |
| CN105427230A (en) | Reversible data hiding method | |
| CN117333398A (en) | Multi-scale image denoising method and device based on self-supervision | |
| CN111723735B (en) | Pseudo high bit rate HEVC video detection method based on convolutional neural network | |
| CN111429363A (en) | Video noise reduction method based on video coding | |
| CN114640857A (en) | Image compression method and 3D noise reduction method based on improved Huffman coding | |
| CN113591681A (en) | Face detection and protection method and device, electronic equipment and storage medium | |
| CN110677624A (en) | Monitoring video-oriented foreground and background parallel compression method based on deep learning | |
| Liang et al. | Multi-scale and multi-patch transformer for sandstorm image enhancement | |
| CN111311698A (en) | Image compression method and system for multi-scale target | |
| CN116012272A (en) | Compressed video quality enhancement method based on reconstructed flow field | |
| Xie et al. | Just noticeable visual redundancy forecasting: a deep multimodal-driven approach | |
| CN111866511B (en) | Video damage repairing method based on convolution long-short term memory neural network | |
| CN114549302A (en) | Image super-resolution reconstruction method and system | |
| CN112509107A (en) | Point cloud attribute recoloring method, device and encoder | |
| CN116634147B (en) | HEVC-SCC intra-frame CU rapid partitioning coding method and device based on multi-scale feature fusion | |
| CN106375768B (en) | Video steganalysis method based on intra prediction mode calibration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200717 |