CN112950491A - Video processing method and device - Google Patents
Video processing method and device Download PDFInfo
- Publication number
- CN112950491A CN112950491A CN202110104898.7A CN202110104898A CN112950491A CN 112950491 A CN112950491 A CN 112950491A CN 202110104898 A CN202110104898 A CN 202110104898A CN 112950491 A CN112950491 A CN 112950491A
- Authority
- CN
- China
- Prior art keywords
- video
- processed
- sharpening
- image distortion
- degree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title abstract description 3
- 238000007906 compression Methods 0.000 claims abstract description 77
- 230000006835 compression Effects 0.000 claims abstract description 75
- 238000012545 processing Methods 0.000 claims abstract description 74
- 238000000034 method Methods 0.000 claims abstract description 43
- 230000009467 reduction Effects 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 9
- 238000005516 engineering process Methods 0.000 description 8
- 238000007781 pre-processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 239000000654 additive Substances 0.000 description 3
- 230000000996 additive effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000006735 deficit Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003707 image sharpening Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Processing (AREA)
Abstract
The application provides a video processing method and device, wherein a to-be-processed video is obtained firstly, then based on preset sharpening strength, sharpening processing and coding compression are carried out on the to-be-processed video to obtain a video after coding compression, then the detail loss degree and the image distortion degree of the video after coding compression are obtained, the detail loss gradient and the image distortion gradient of the to-be-processed video are calculated based on the detail loss degree and the image distortion degree, then based on the detail loss gradient and the image distortion gradient, the edge signal-to-noise ratio of the to-be-processed video is determined, finally based on the edge signal-to-noise ratio, the optimal sharpening strength of the to-be-processed video is determined, and the to-be-processed video is sharpened. The method correlates the sharpening processing and the coding compression of the video, and selects the optimal sharpening strength of the video to sharpen and then to code compress, thereby achieving better coding compression effect than the existing coding compression mode.
Description
Technical Field
The application relates to the technical field of computer video processing, in particular to a video processing technology.
Background
With the increasing demand of people on the quality of video images, more and more ultrahigh-definition videos are obtained by adopting video technologies such as 4k, 8k, hdr and the like, and various types and different-quality videos are also produced by the development of various computer video applications, so that the videos bring huge pressure on computer storage resources and transmission network bandwidth, and strong demands and challenges are brought to video processing and coding compression technologies.
Before a computer stores or transmits video, the video is usually encoded and compressed in order to save computer storage resources and transmission bandwidth. The video coding compression is lossy coding compression, which usually causes inevitable loss of image details and image distortion, and affects the quality of the main and guest objects after coding compression.
In order to reduce the loss degree of the video caused by video coding compression and improve the quality of the compressed video, various video image enhancement technologies can be adopted to preprocess the video before the video coding compression, and then the video coding compression is carried out.
The current mainstream video preprocessing technology comprises noise reduction, sharpening, multi-frame quality enhancement and the like, but the existing video preprocessing technology adopts a video processing mode based on the same parameter for the same video, and lacks correlation with video coding compression adopted subsequently, and cannot adjust the parameter of the previous video preprocessing by combining the compression effect of the subsequent video coding. .
Disclosure of Invention
The present application aims to provide a method and an apparatus for video processing, so as to solve the technical problem in the prior art that the parameter setting of video preprocessing is not combined with the subsequent video coding compression effect.
According to an aspect of the present application, there is provided a method of video processing, wherein the method comprises:
acquiring a video to be processed;
based on preset sharpening strength, sharpening the video to be processed, and carrying out coding compression on the sharpened video to obtain a coded and compressed video;
acquiring the detail loss degree and the image distortion degree of the video subjected to encoding compression, and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the video subjected to encoding compression;
determining an edge signal-to-noise ratio of the video to be processed based on the detail loss gradient and the image distortion gradient;
and determining the optimal sharpening strength of the video to be processed based on the edge signal-to-noise ratio of the video to be processed, and sharpening the video to be processed based on the optimal sharpening strength.
Optionally, before the obtaining the video to be processed, the method further includes:
acquiring an original video;
and dividing the original video into a plurality of videos to be processed based on the scenes of each frame of video image of the original video, wherein each frame of video image of the videos to be processed contains the same scene.
Optionally, the obtaining the detail loss degree and the image distortion degree of the encoded and compressed video includes:
acquiring the detail loss degree and the image distortion degree of each frame of video image of the coded and compressed video;
calculating the average value of the detail loss degree of each frame of video image and the average value of the image distortion degree of each frame of video image;
and determining the average value of the detail loss degrees of the video images of the frames as the detail loss degree of the video after the coding compression, and determining the average value of the image distortion degrees of the video images of the frames as the image distortion degree of the video after the coding compression.
Optionally, the sharpening the to-be-processed video based on the preset sharpening strength, and performing encoding compression on the sharpened video to obtain an encoded and compressed video includes:
based on a first preset sharpening intensity and a second preset sharpening intensity respectively, sharpening the video to be processed, and coding and compressing the sharpened video respectively to obtain a first video and a second video which are coded and compressed;
wherein the obtaining the detail loss degree and the image distortion degree of the video after the encoding and the compression, and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the video after the encoding and the compression comprises:
respectively obtaining the detail loss degree and the image distortion degree of the first video and the second video;
and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the first video and the second video.
Optionally, wherein the calculating of the detail loss gradient of the video to be processed includes:
and subtracting the detail loss degree of the second video from the detail loss degree of the first video.
Optionally, wherein the calculating of the image distortion gradient of the video to be processed includes:
and subtracting the image distortion degree of the second video from the image distortion degree of the first video to obtain a numerical value.
Optionally, wherein the first preset sharpening intensity is 1 and the second preset sharpening intensity is 10.
Optionally, the method for video processing further includes:
and carrying out noise reduction processing on the video to be processed after sharpening processing based on the optimal sharpening strength.
Optionally, the method for video processing further includes:
and coding and compressing the processed video to be processed to obtain the coded and compressed video.
According to another aspect of the present application, there is also provided an apparatus for video processing, wherein the apparatus comprises:
the second device is used for acquiring a video to be processed, wherein each frame of video image of the video to be processed contains the same scene;
the third device is used for carrying out sharpening processing on the video to be processed based on preset sharpening strength and carrying out coding compression on the sharpened video to be processed so as to obtain a coded and compressed video;
the fourth device is used for acquiring the detail loss degree and the image distortion degree of the video after the coding compression, and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the video after the coding compression;
a fifth means for determining an edge signal-to-noise ratio of the video to be processed based on the detail loss gradient and the image distortion gradient;
and the sixth device is used for determining the optimal sharpening strength of the video to be processed based on the edge signal-to-noise ratio of the video to be processed and sharpening the video to be processed based on the optimal sharpening strength.
Optionally, the apparatus for video processing further includes:
the first device is used for acquiring an original video and dividing the original video into a plurality of videos to be processed based on the scene of each frame of video image of the original video.
Optionally, the apparatus for video processing further includes:
and the seventh device is used for carrying out noise reduction processing on the video to be processed after the sharpening processing based on the optimal sharpening strength.
Optionally, the apparatus for video processing further includes:
and the eighth device is used for coding and compressing the processed video to be processed to obtain the coded and compressed video.
Compared with the prior art, the method and the device for processing the video are provided, firstly, the video to be processed is obtained, then based on the preset sharpening strength, sharpening the video to be processed, and encoding and compressing the sharpened video to obtain an encoded and compressed video, then, the detail loss degree and the image distortion degree of the video after the coding compression are obtained, the detail loss gradient and the image distortion gradient of the video to be processed are calculated based on the detail loss degree and the image distortion degree of the video after the coding compression, the edge signal-to-noise ratio of the video to be processed is determined based on the detail loss gradient and the image distortion gradient, finally, the optimal sharpening strength of the video to be processed is determined based on the edge signal-to-noise ratio of the video to be processed, and the sharpening processing is carried out on the video to be processed based on the optimal sharpening strength. The method correlates the sharpening processing before the video coding compression with the video coding compression, and selects the optimal sharpening strength of the video to sharpen and then carry out the coding compression, thereby achieving better coding compression effect than the existing coding compression mode, and improving the quality of the compressed video to the maximum extent on the basis of further reducing the video code rate.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
FIG. 1 illustrates a flow diagram of a method of video processing according to an aspect of the subject application;
FIG. 2 shows a schematic diagram of a video processing apparatus according to another aspect of the present application;
the same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the present application, each module and trusted party of the system includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
In order to further explain the technical means and effects adopted by the present application, the following description clearly and completely describes the technical solution of the present application with reference to the accompanying drawings and preferred embodiments.
FIG. 1 illustrates a flow diagram of a method of video processing in one aspect of the subject application, wherein the method of an embodiment comprises:
s12, acquiring a video to be processed;
s13, based on preset sharpening strength, sharpening the video to be processed, and coding and compressing the sharpened video to obtain a coded and compressed video;
s14, acquiring the detail loss degree and the image distortion degree of the video after encoding and compressing, and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the video after encoding and compressing;
s15, determining the edge signal-to-noise ratio of the video to be processed based on the detail loss gradient and the image distortion gradient;
s16, based on the edge signal-to-noise ratio of the video to be processed, determining the optimal sharpening strength of the video to be processed, and based on the optimal sharpening strength, sharpening the video to be processed.
In the present application, the method is performed by a device 1, the device 1 is a computer device and/or a cloud, the computer device includes but is not limited to a personal computer, a notebook computer, an industrial computer, a network host, a single network server, a plurality of network server sets; the Cloud is made up of a large number of computers or web servers based on Cloud Computing (Cloud Computing), which is a type of distributed Computing, a virtual supercomputer consisting of a collection of loosely coupled computers.
The computer device and/or cloud are merely examples, and other existing or future devices and/or resource sharing platforms, as applicable to the present application, are also intended to be included within the scope of the present application and are hereby incorporated by reference.
In this embodiment, in step S12, if the device 1 is provided with a video capture device, the video to be processed may be obtained through the video capture device provided by the device 1, or if the device 1 is not integrated with the video capture device, the video to be processed may also be obtained through a network or a copy mode. The manner in which the apparatus 1 acquires the video to be processed is not limited, and any acquisition manner, if applicable, should be included in the scope of the present application.
Sharpening a video image is a common video preprocessing technique. According to the characteristics of video image sharpening and the requirement of better coding and compressing effects, the effect obtained by carrying out coding and compressing after sharpening the videos with the same scene is better than the effect obtained by carrying out coding and compressing after sharpening the videos with different scenes by adopting the same sharpening strength.
In this embodiment, optionally, the method for video processing further includes:
s110 (not shown) acquires an original video;
s111 (not shown) segments the original video into a plurality of videos to be processed based on a scene of each frame of video image of the original video, where each frame of video image of the videos to be processed includes the same scene.
In step S110, the original video is obtained through a video capture device of the apparatus 1, or the original video may be obtained through a network. The manner in which the original video is obtained by the apparatus 1 is not limited, and any obtaining manner, as applicable, is also included in the scope of the present application.
In step S111, the device 1 analyzes the acquired original video frame by frame, and divides the original video into a plurality of videos to be processed based on the scene of each frame of video image of the acquired original video, where each frame of video image of the videos to be processed includes the same scene.
For example, the gray values of the video images of the adjacent frames in the original video may be obtained, MSE (mean square error) is used to compare the similarity of the gray values between the video images of the adjacent frames, and if the similarity exceeds a preset threshold, it is determined that scene change occurs.
In this embodiment, in step S13, the device 1 performs sharpening on the to-be-processed video based on a preset sharpening strength, and performs encoding and compressing on the sharpened video to obtain an encoded and compressed video. The encoding compression method is not particularly limited in the present application, and may be an MPEG method or an h.26x method, and any acquisition method, if applied to the present application, is also included in the scope of the present application.
In this embodiment, in step S14, the apparatus 1 obtains the detail loss degree and the image distortion degree of the encoded and compressed video, and calculates the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the encoded and compressed video.
The loss of detail (dl) of the video image refers to the loss of detail of the video image, which reflects the loss of useful visual information affecting the visibility of the content and the degree of additive impairment affecting the viewing experience (referring to redundant visual information, i.e., information appearing in the test image that distracts the viewer from the useful content, thereby resulting in a poor viewing experience, such as image noise enhanced by sharpening).
dl is proposed in the IEEE paper Image Quality Assessment by separation Evaluating Detail fields and Additive images published in 2011. The smaller the dl, the higher the degree of texture detail retention of the video image, and the better the video image quality.
The dl after the video sharpening processing and the encoding compression can be obtained by calculating the difference and additive damage of useful visual information between each frame of video image in the video and each frame of video image after the sharpening processing and the video encoding compression are performed under the same sharpening condition, and then the dl after the video sharpening processing and the encoding compression can be obtained, for example, the dl after the sharpening processing and the encoding compression of each frame of video image in the video can be calculated through an ADM algorithm in a VMAF (video quality measurement tool) of Netflix (Netflix), and then the dl after the video sharpening processing and the encoding compression can be obtained. The method can obtain the detail loss gradient (g _ dl) of the video after sharpening processing and encoding compression by respectively obtaining the dl of the same video after sharpening processing and encoding compression under different sharpening conditions so as to evaluate the influence degree of the same video after sharpening processing and encoding compression under different sharpening conditions on the detail loss of the video image.
The video distortion (vl) of the video image refers to the distortion degree of the video image, and is obtained by comparing a distorted video image obtained by processing an original video image with the original video image according to a Human Visual System (HVS). The information retention degree of the video image after image enhancement processing such as sharpening and lossy compression is reflected.
Algorithms are usually used to compare the difference between the original Video image and the distorted Video image after sharpening and encoding compression, for example, vl after sharpening and encoding compression can be calculated by VIF algorithm in VMAF (Video Multi-method Assessment Fusion) which is a Video quality measurement tool of neffix (Netflix), and the algorithms such as MSE, PSNR, SSIM, MS-SSIM, JND can also be used.
The larger vl is, the higher the information loss degree of the video image is, and the worse the video image quality is.
The difference between each frame of video image in the video and each frame of video image subjected to sharpening processing and video coding compression under the same sharpening condition can be calculated to obtain vl after each frame of video image is sharpened and coded and compressed, and then vl after the video sharpening processing and the video coding compression can be obtained. The sharpening processing and the vl after the coding compression of the same video under different sharpening conditions can be respectively obtained to obtain the image distortion gradient (g _ vl) after the video sharpening processing and the coding compression, so as to evaluate the influence degree of the same video on the video image distortion after the sharpening processing and the video coding compression under different sharpening conditions.
Optionally, the obtaining the detail loss degree and the image distortion degree of the encoded and compressed video includes:
acquiring the detail loss degree and the image distortion degree of each frame of video image of the coded and compressed video;
calculating the average value of the detail loss degree of each frame of video image and the average value of the image distortion degree of each frame of video image;
and determining the average value of the detail loss degrees of the video images of the frames as the detail loss degree of the video after the coding compression, and determining the average value of the image distortion degrees of the video images of the frames as the image distortion degree of the video after the coding compression.
In the embodiment, for example, the video to be processed includes N frames of video images { f1, f2, …, fN }, sharpening processing is performed on each frame of video image with a preset sharpening intensity S, dl corresponding to each frame of video image after being compressed is coded and is dl1 to dlN, and vl corresponding to each frame of video image is vl1 to vlN, so that dl of the video to be processed may be an average value of dl1 to dlN corresponding to each frame of video image included in the video, such as an arithmetic average value:
the vl of the video to be processed may be an average value of vl1 to vlN corresponding to each frame of video image included in the video, such as an arithmetic average value:
the average value of the detail loss degrees of the video images of the frames may also be a geometric average value of the detail loss degrees of the video images of the frames, or a root-mean-square average value of the detail loss degrees of the video images of the frames. The average value of the image distortion degrees of the video images of the frames can also be a geometric average value of the image distortion degrees of the video images of the frames, and can also be a root mean square average value of the image distortion degrees of the video images of the frames.
Optionally, wherein the step S13 includes:
based on a first preset sharpening intensity and a second preset sharpening intensity respectively, sharpening the video to be processed, and coding and compressing the sharpened video respectively to obtain a first video and a second video which are coded and compressed;
wherein the step S14 includes:
respectively obtaining the detail loss degree and the image distortion degree of the first video and the second video;
and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the first video and the second video.
In the embodiment, for example, the video to be processed is sharpened at a first preset sharpening intensity S0, and is encoded and compressed to obtain a first video, where a corresponding detail loss degree is dlS0After the second preset sharpening intensity S1, the second video is obtained after the second video is subjected to sharpening processing and coded and compressed, and the corresponding detail loss degree is dlS1Then can be based on dlS0And dlS1And calculating a detail loss gradient g _ dl of the video to be processed. Similarly, the video to be processed is sharpened with a first preset sharpening strength S0, and is encoded and compressed to obtain a corresponding image distortion vl of the first videoS0After the sharpening process with the second preset sharpening strength S1 is performed, and the image distortion degree vl corresponding to the second video is obtained after the encoding and the compressionS1Then can be based on vlS0And vlS1And calculating the detail loss gradient g _ vl of the video to be processed.
Optionally, wherein the calculating of the detail loss gradient of the video to be processed includes:
and subtracting the detail loss degree of the second video from the detail loss degree of the first video.
For example, g _ dl of the to-be-processed video may be dl corresponding to the first video obtained by subjecting the to-be-processed video to sharpening processing with a first preset sharpening strength S0, and encoding and compressing the to-be-processed videoS0The dl corresponding to the second video is obtained after the second preset sharpening strength S1 is performed on the second video and the second video is coded and compressedS1The difference of (a) is:
g_dl＝dlS0-dlS1
optionally, wherein the calculating of the image distortion gradient of the video to be processed includes:
and subtracting the image distortion degree of the second video from the image distortion degree of the first video to obtain a numerical value.
Such asThe g _ vl of the to-be-processed video may be obtained by subjecting the to-be-processed video to sharpening processing with a first preset sharpening strength S0, and encoding and compressing the to-be-processed video to obtain vl corresponding to the first videoS0Obtaining vl corresponding to the second video after the sharpening processing with the second preset sharpening strength S1 and the coding compressionS1The difference of (a) is:
g_vl＝vlS0-vlS1
generally, sharpening is carried out on a video to be processed, the selection range of sharpening strength is 0-10, wherein 0 is selected to mean that no sharpening is carried out.
Optionally, wherein the first preset sharpening intensity is 1 and the second preset sharpening intensity is 10.
Continuing in this embodiment, in step S15, the apparatus 1 determines an edge snr of the video to be processed based on the detail loss gradient and the image distortion gradient.
Wherein, after determining g _ dl and g _ vl of the video to be processed under the first preset sharpening strength and the second preset sharpening strength, an edge signal-to-noise ratio (ENR) of the video to be processed can be determined, that is, ENR is a function of g _ dl and g _ vl,
ENR＝f(g_dl,g_vl)
this function is related to the resolution of the video to be processed. For example, if the resolution of the video to be processed is 1080P, the expression of f (g _ dl, g _ vl) can be defined as follows:
wherein, alpha1～ɑ4Is a hyper-parameter related to the resolution of the video to be processed.
In this embodiment, in step S16, the device 1 determines an optimal sharpening strength of the to-be-processed video based on the edge signal-to-noise ratio of the to-be-processed video, and performs a sharpening process on the to-be-processed video based on the optimal sharpening strength.
The method comprises the following steps of determining the sharpening strength which should be adopted for a video to be processed according to the ENR of the determined video to be processed: when ENR is 0, the influence of noise representing the video to be processed on the quality of the video after the video is coded and compressed after sharpening processing can be ignored, so that the video to be processed can be sharpened by adopting higher sharpening strength; by analogy, when ENR is 3, it indicates that quality degradation brought by the noise of the video to be processed after sharpening to the video coding compression is greater than quality improvement brought by sharpening, so that it is necessary to adopt lower sharpening strength or perform sharpening and coding compression after performing denoising processing on the video to balance the influence of details and noise on the video coding compression quality.
For example, based on the collected data representation of various types of videos to be processed, for a 1080P video to be processed, ENRs at different g _ dl and g _ vl, and the determined corresponding optimal sharpening strengths are shown in table 1.
TABLE 1
| g_dl | g_vl | ENR | Optimum sharpening Strength (0-10) |
| [ɑ1，+∞) | (-∞,ɑ0] | 0 | 9 |
| [ɑ2，ɑ1) | (ɑ0,+∞] | 1 | 8 |
| [ɑ3，ɑ2) | (ɑ0，+∞] | 2 | 6 |
| [-∞，ɑ3) | (ɑ0，+∞] | 3 | 4 |
Continuing the above embodiment, after the above steps, if the computed ENR is 0 for a to-be-processed video with a resolution of 1080P, it indicates that the noise of the to-be-processed video after sharpening has negligible impact on the quality of the video after encoding and compressing, referring to table 1, the to-be-processed video may be sharpened with an optimal sharpening strength of 9 to obtain the optimal detail enhancement and encoding and compressing effects; if ENR is calculated to be 3, it indicates that quality degradation brought by noise of the video to be processed after sharpening processing to the video coded and compressed is larger than quality improvement brought by sharpening processing, referring to table 1, sharpening processing needs to be performed on the video to be processed with an optimal sharpening strength of 4, so as to balance influence of details and noise on video coded and compressed quality.
Optionally, the method for video processing further includes:
s17 (not shown) performs noise reduction processing on the video to be processed after sharpening processing based on the optimal sharpening strength.
In the above embodiment, the ENR of the video to be processed is determined by calculating g _ dl and g _ vl of the video to be processed under different sharpening parameters, so as to determine the optimal sharpening strength, so as to balance the influence of high-frequency noise on subsequent video coding compression.
Continuing with the above embodiment, the device 1 may further perform noise reduction processing on the video to be processed after sharpening processing based on the optimal sharpening strength, so as to balance the influence of subsequent video coding compression by other spectral noise.
Optionally, the method for video processing further includes:
s18 (not shown) performs encoding compression on the processed video to be processed to obtain an encoded compressed video.
Fig. 2 shows a schematic diagram of a video processing apparatus according to another aspect of the present application, wherein the apparatus comprises:
a second device 22, configured to obtain a video to be processed;
a third device 23, configured to perform sharpening on the to-be-processed video based on a preset sharpening strength, and perform encoding compression on the to-be-processed video after sharpening to obtain an encoded and compressed video;
a fourth device 24, configured to obtain a detail loss degree and an image distortion degree of the coded and compressed video, and calculate a detail loss gradient and an image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the coded and compressed video;
a fifth means 25 for determining an edge snr of the video to be processed based on the detail loss gradient and the image distortion gradient;
a sixth device 26, configured to determine an optimal sharpening strength of the video to be processed based on an edge signal-to-noise ratio of the video to be processed, and perform sharpening on the video to be processed based on the optimal sharpening strength.
In this embodiment, the apparatus is identical to the apparatus 1 described hereinbefore.
Wherein, the second device 22 of the apparatus first obtains a video to be processed, then the third module 23 of the apparatus performs sharpening processing on the video to be processed based on a preset sharpening strength, and performs encoding compression on the sharpened video to be processed to obtain an encoded and compressed video, then the fourth module 24 of the apparatus obtains a detail loss degree and an image distortion degree of the encoded and compressed video, and calculates a detail loss gradient and an image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the encoded and compressed video, the fifth device 25 of the apparatus determines an edge signal-to-noise ratio of the video to be processed based on the detail loss gradient and the image distortion gradient, and finally the sixth device 26 of the apparatus determines an optimal sharpening strength of the video to be processed based on the edge signal-to-noise ratio of the video to be processed, and based on the optimal sharpening strength, carrying out sharpening processing on the video to be processed.
Optionally, the apparatus for video processing further includes:
a first device 21 (not shown) for acquiring an original video and dividing the original video into a plurality of videos to be processed based on a scene of each frame of video image of the original video.
The first device 21 of the apparatus acquires an original video, divides the original video into a plurality of videos to be processed according to scenes of each frame of video image of the original video, the second device 22 of the apparatus acquires the videos to be processed one by one, and subsequent devices of the apparatus process the videos to be processed.
Optionally, the apparatus for video processing further includes:
seventh means 27 (not shown) for performing noise reduction processing on the video to be processed after sharpening processing based on the optimal sharpening strength.
Optionally, the apparatus for video processing further includes:
eighth means 28 (not shown) for performing encoding compression on the processed video to be processed to obtain an encoded compressed video.
According to yet another aspect of the present application, there is also provided a computer readable medium having stored thereon computer readable instructions executable by a processor to implement the foregoing method.
According to yet another aspect of the present application, there is also provided an apparatus for video processing, wherein the apparatus comprises:
one or more processors; and
a memory storing computer readable instructions that, when executed, cause the processor to perform operations of the method as previously described.
For example, the computer readable instructions, when executed, cause the one or more processors to: acquiring a video to be processed; based on preset sharpening strength, sharpening the video to be processed, and carrying out coding compression on the sharpened video to obtain a coded and compressed video; acquiring the detail loss degree and the image distortion degree of the video subjected to encoding compression, and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the video subjected to encoding compression; determining an edge signal-to-noise ratio of the video to be processed based on the detail loss gradient and the image distortion gradient; and determining the optimal sharpening strength of the video to be processed based on the edge signal-to-noise ratio of the video to be processed, and sharpening the video to be processed based on the optimal sharpening strength.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (15)
1. A method for video processing, the method comprising:
acquiring a video to be processed;
based on preset sharpening strength, sharpening the video to be processed, and carrying out coding compression on the sharpened video to obtain a coded and compressed video;
acquiring the detail loss degree and the image distortion degree of the video subjected to encoding compression, and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the video subjected to encoding compression;
determining an edge signal-to-noise ratio of the video to be processed based on the detail loss gradient and the image distortion gradient;
and determining the optimal sharpening strength of the video to be processed based on the edge signal-to-noise ratio of the video to be processed, and sharpening the video to be processed based on the optimal sharpening strength.
2. The method of claim 1, wherein prior to said obtaining the video to be processed, the method further comprises:
acquiring an original video;
and dividing the original video into a plurality of videos to be processed based on the scenes of each frame of video image of the original video, wherein each frame of video image of the videos to be processed contains the same scene.
3. The method according to claim 1 or 2, wherein the obtaining the detail loss degree and the image distortion degree of the encoded and compressed video comprises:
acquiring the detail loss degree and the image distortion degree of each frame of video image of the coded and compressed video;
calculating the average value of the detail loss degree of each frame of video image and the average value of the image distortion degree of each frame of video image;
and determining the average value of the detail loss degrees of the video images of the frames as the detail loss degree of the video after the coding compression, and determining the average value of the image distortion degrees of the video images of the frames as the image distortion degree of the video after the coding compression.
4. The method of claim 3, wherein the dividing the original video into the plurality of videos to be processed based on the scenes of each frame of video image of the original video, wherein each frame of video image of the videos to be processed contains the same scenes comprises:
based on a first preset sharpening intensity and a second preset sharpening intensity respectively, sharpening the video to be processed, and coding and compressing the sharpened video respectively to obtain a first video and a second video which are coded and compressed;
wherein the obtaining the detail loss degree and the image distortion degree of the video after the encoding and the compression, and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the video after the encoding and the compression comprises:
respectively obtaining the detail loss degree and the image distortion degree of the first video and the second video;
and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the first video and the second video.
5. The method of claim 4, wherein the calculating the detail loss gradient of the video to be processed comprises:
and subtracting the detail loss degree of the second video from the detail loss degree of the first video.
6. The method of claim 4, wherein the calculating of the image distortion gradient of the video to be processed comprises:
and subtracting the image distortion degree of the second video from the image distortion degree of the first video to obtain a numerical value.
7. The method of claim 5 or 6, wherein the first preset sharpening intensity is 1 and the second preset sharpening intensity is 10.
8. The method according to any one of claims 1 to 7, further comprising:
and carrying out noise reduction processing on the video to be processed after sharpening processing based on the optimal sharpening strength.
9. The method of claims 1 to 8, further comprising:
and coding and compressing the processed video to be processed to obtain the coded and compressed video.
10. An apparatus for video processing, the apparatus comprising:
the second device is used for acquiring a video to be processed;
the third device is used for carrying out sharpening processing on the video to be processed based on preset sharpening strength and carrying out coding compression on the sharpened video to be processed so as to obtain a coded and compressed video;
the fourth device is used for acquiring the detail loss degree and the image distortion degree of the video after the coding compression, and calculating the detail loss gradient and the image distortion gradient of the video to be processed based on the detail loss degree and the image distortion degree of the video after the coding compression;
a fifth means for determining an edge signal-to-noise ratio of the video to be processed based on the detail loss gradient and the image distortion gradient;
and the sixth device is used for determining the optimal sharpening strength of the video to be processed based on the edge signal-to-noise ratio of the video to be processed and sharpening the video to be processed based on the optimal sharpening strength.
11. The apparatus of claim 10, further comprising:
the first device is used for acquiring an original video and dividing the original video into a plurality of videos to be processed based on the scene of each frame of video image of the original video.
12. The apparatus according to claim 10 or 11, characterized in that it further comprises:
and the seventh device is used for carrying out noise reduction processing on the video to be processed after the sharpening processing based on the optimal sharpening strength.
13. The apparatus according to any one of claims 10 to 12, characterized in that it further comprises:
and the eighth device is used for coding and compressing the processed video to be processed to obtain the coded and compressed video.
14. A computer-readable medium comprising, in combination,
stored thereon computer readable instructions to be executed by a processor to implement the method of any one of claims 1 to 9.
15. An apparatus for video processing, the apparatus comprising:
one or more processors; and
a memory storing computer readable instructions that, when executed, cause the processor to perform the operations of the method of any of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110104898.7A CN112950491B (en) | 2021-01-26 | 2021-01-26 | Video processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110104898.7A CN112950491B (en) | 2021-01-26 | 2021-01-26 | Video processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112950491A true CN112950491A (en) | 2021-06-11 |
| CN112950491B CN112950491B (en) | 2024-02-13 |
Family
ID=76237136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110104898.7A Active CN112950491B (en) | 2021-01-26 | 2021-01-26 | Video processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112950491B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113507643A (en) * | 2021-07-09 | 2021-10-15 | Oppo广东移动通信有限公司 | Video processing method, device, terminal and storage medium |
| WO2023274005A1 (en) * | 2021-06-30 | 2023-01-05 | 北京字跳网络技术有限公司 | Image processing method and apparatus, electronic device, and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809748A (en) * | 2015-05-13 | 2015-07-29 | 西安电子科技大学 | Image compression sensing method based on variable sampling rate and linear mean prediction |
| US20180220162A1 (en) * | 2015-09-25 | 2018-08-02 | Huawei Technologies Co., Ltd. | Adaptive sharpening filter for predictive coding |
| CN109934785A (en) * | 2019-03-12 | 2019-06-25 | 湖南国科微电子股份有限公司 | Image sharpening method and device |
| CN110111261A (en) * | 2019-03-28 | 2019-08-09 | 福州瑞芯微电子股份有限公司 | Adaptive equalization processing method, electronic equipment and the computer readable storage medium of image |
| CN110267045A (en) * | 2019-08-07 | 2019-09-20 | 杭州微帧信息科技有限公司 | A kind of method, apparatus and readable storage medium storing program for executing that video is handled and encoded |
| CN111667410A (en) * | 2020-06-10 | 2020-09-15 | 腾讯科技（深圳）有限公司 | Image resolution improving method and device and electronic equipment |
-
2021
- 2021-01-26 CN CN202110104898.7A patent/CN112950491B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809748A (en) * | 2015-05-13 | 2015-07-29 | 西安电子科技大学 | Image compression sensing method based on variable sampling rate and linear mean prediction |
| US20180220162A1 (en) * | 2015-09-25 | 2018-08-02 | Huawei Technologies Co., Ltd. | Adaptive sharpening filter for predictive coding |
| CN109934785A (en) * | 2019-03-12 | 2019-06-25 | 湖南国科微电子股份有限公司 | Image sharpening method and device |
| CN110111261A (en) * | 2019-03-28 | 2019-08-09 | 福州瑞芯微电子股份有限公司 | Adaptive equalization processing method, electronic equipment and the computer readable storage medium of image |
| CN110267045A (en) * | 2019-08-07 | 2019-09-20 | 杭州微帧信息科技有限公司 | A kind of method, apparatus and readable storage medium storing program for executing that video is handled and encoded |
| CN111667410A (en) * | 2020-06-10 | 2020-09-15 | 腾讯科技（深圳）有限公司 | Image resolution improving method and device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 张聪等: "噪声控制的图像非线性锐化方法", 计算机工程与应用, vol. 47, no. 04, 24 January 2011 (2011-01-24), pages 154 - 156 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023274005A1 (en) * | 2021-06-30 | 2023-01-05 | 北京字跳网络技术有限公司 | Image processing method and apparatus, electronic device, and storage medium |
| CN113507643A (en) * | 2021-07-09 | 2021-10-15 | Oppo广东移动通信有限公司 | Video processing method, device, terminal and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112950491B (en) | 2024-02-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sara et al. | Image quality assessment through FSIM, SSIM, MSE and PSNR—a comparative study | |
| Lee et al. | Contrast enhancement based on layered difference representation of 2D histograms | |
| EP2950267B1 (en) | Image denoising method and image denoising apparatus | |
| US9811882B2 (en) | Method and apparatus for processing super resolution image using adaptive preprocessing filtering and/or postprocessing filtering | |
| CN108090886B (en) | High dynamic range infrared image display and detail enhancement method | |
| US6823089B1 (en) | Method of determining the extent of blocking and contouring artifacts in a digital image | |
| CN112950491B (en) | Video processing method and device | |
| US20070195106A1 (en) | Detecting Doctored JPEG Images | |
| WO2017190691A1 (en) | Picture compression method and apparatus | |
| US6643410B1 (en) | Method of determining the extent of blocking artifacts in a digital image | |
| Hossain et al. | Medical image enhancement based on nonlinear technique and logarithmic transform coefficient histogram matching | |
| CN107886518B (en) | Picture detection method and device, electronic equipment and readable storage medium | |
| WO2018225133A1 (en) | Image processing device, image processing method and image processing program | |
| CN111161177B (en) | Image self-adaptive noise reduction method and device | |
| Zhang et al. | Adaptive compressed sensing of color images based on salient region detection | |
| KR20220043764A (en) | Method and apparatus for video quality assessment | |
| Liu et al. | High dynamic range image quality assessment based on frequency disparity | |
| CN111311584B (en) | Video quality evaluation method and device, electronic equipment and readable medium | |
| Sun et al. | Readability enhancement of low light videos based on discrete wavelet transform | |
| Gunawan et al. | A review on high dynamic range (HDR) image quality assessment | |
| O’Byrne et al. | Impact of video compression on the performance of object detection systems for surveillance applications | |
| Pal et al. | Visibility enhancement of fog degraded images using adaptive defogging function | |
| Srivastava et al. | Adaptive image compression using saliency and KAZE features | |
| CN113038129A (en) | Method and equipment for acquiring data samples for machine learning | |
| CN114827567A (en) | Video quality analysis method, apparatus and readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |