WO2019037471A1

WO2019037471A1 - Video processing method, video processing device and terminal

Info

Publication number: WO2019037471A1
Application number: PCT/CN2018/086167
Authority: WO
Inventors: 张海婷; 邵红胜
Original assignee: 中兴通讯股份有限公司
Priority date: 2017-08-24
Filing date: 2018-05-09
Publication date: 2019-02-28
Also published as: CN109429098A

Abstract

Provided by the present application are a video processing method, video processing device and terminal. The method comprises: decoding a video to be proceeded so as to obtain an image set consisting of a plurality of video images; replacing a predetermined video image with a sample image, wherein the predetermined video image is a video image in the image set which has a degree of similarity to the sample image greater than a preset similarity threshold; and encoding the image set having the plurality of video images after replacing the predetermined video image with the sample image so as to obtain a processed video.

Description

Video processing method, video processing device, and terminal

Technical field

The present disclosure relates to the field of image processing, and in particular, to a video processing method, a video processing apparatus, and a terminal.

Background technique

Video has different applications in every aspect. For example, panoramic video can be applied to tourist attraction promotion, panoramic map production, advertising entertainment promotion, virtual reality technology, and the like. However, due to the inconsistency of the equipment for shooting video, the quality of the captured video may be uneven, which cannot meet the requirements of publicity such as tourist attractions.

Therefore, there is a need for a video processing method to improve video quality.

Summary of the invention

The present disclosure provides a video processing method, a video processing device, and a terminal.

According to a first aspect of the present disclosure, there is provided a video processing method comprising: decoding a video to be processed to obtain a picture set composed of a plurality of video pictures; replacing the predetermined video picture with an example picture, wherein the predetermined video The picture is a video picture in which the similarity between the picture set and the example picture is greater than a preset similarity threshold; and the picture set after replacing the predetermined video picture with the example picture is encoded to obtain a processed picture. video.

According to a second aspect of the present disclosure, there is also provided a video processing apparatus comprising: a decoding module configured to decode a video to be processed to obtain a picture set composed of a plurality of video pictures; a replacement module configured to place the predetermined video The picture is replaced with an example picture, wherein the predetermined video picture is a video picture in which the similarity of the picture set to the example picture is greater than a preset similarity threshold; and an encoding module configured to replace with the example picture The set of pictures after the predetermined video picture is encoded to obtain a processed video.

According to a third aspect of the present disclosure, there is also provided a terminal comprising: a video processing device as described above.

According to a fourth aspect of the present disclosure, there is also provided a computer readable storage medium having stored therein a computer executable program, wherein when the computer executable program is run on a processor, The following steps may be performed: decoding a video to be processed to obtain a picture set composed of multiple video pictures; replacing the predetermined video picture with an example picture, wherein the predetermined video picture is that the similarity between the picture set and the example picture is greater than the similarity threshold a video picture; and encoding a picture set after replacing the predetermined video picture with the example picture to obtain a processed video.

DRAWINGS

1 is a flowchart of a video processing method in accordance with an embodiment of the present disclosure;

2 is a schematic structural diagram of a video processing device according to an embodiment of the present disclosure.

Detailed ways

In order to solve the problem that the video quality of the prior art is uneven and cannot fully satisfy the practical application, the present disclosure provides a video processing method, a video processing apparatus, and a terminal. The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the disclosure and are not intended to be limiting.

According to an embodiment of the present disclosure, a video processing method is provided. As shown in FIG. 1, the method includes the following steps:

S101. Decode a video to be processed to obtain a picture set composed of multiple video pictures.

S102. Replace a predetermined video picture with an example picture, where the predetermined video picture is a video picture whose similarity in the picture set is greater than the similarity threshold of the example picture;

S103. Encode a picture set after replacing the predetermined video picture with the example picture to obtain a processed video.

In the embodiment of the present disclosure, the video to be processed may be a normal video or a panoramic video that requires higher video quality. When encoding and decoding a video to be processed, the video to be processed can be processed using various methods of existing lossless compression or lossy compression. Here, the codec techniques used include, but are not limited to, the following techniques: predictive coding techniques, quantized coding techniques, Shannon coding techniques, and Huffman coding. In actual operation, an appropriate codec technology can be selected according to the video capturing device and the specific application of the video. It should be noted that the method of decoding and encoding the video should be consistent. For example, in the case of decoding a video segment by Huffman coding techniques, it is also necessary to encode the video picture using Huffman coding techniques after being replaced with the example picture.

Before replacing the predetermined video picture with the example picture, it is necessary to take a picture in the same scene as the video to be processed as an example picture. In order to achieve better results, try to take the example picture in the same perspective as the video to be processed, and try to take the corresponding sample picture where the scene changes greatly. At the same time, taking high-quality, high-resolution pictures as an example picture as much as possible can make the processed video quality better.

In order to ensure that the definition of the video can be increased by replacing the example picture into the video, after decoding the video to be processed and obtaining a plurality of video pictures, it can be detected whether the resolution of the predetermined video picture is smaller than the resolution of the sample picture. In the case where the resolution of the predetermined video picture is smaller than the resolution of the example picture, the predetermined video picture is replaced with the example picture. In an exemplary embodiment, the example picture is segmented into example picture blocks of a predetermined size while the predetermined video picture is also segmented into video picture blocks of a predetermined size. The size of each of the divided picture blocks is the same as the size of each of the divided video picture blocks. The similarity between the example picture block and the video picture block is calculated. In the case where the similarity is greater than the preset similarity threshold, the video picture block is replaced with an example picture block. It should be understood that the calculation of the similarity between pictures may be, but is not limited to, the following methods: histogram matching method, matrix decomposition method, and image similarity calculation method based on feature points.

In order to improve the quality of the processed video to achieve a better display effect, after replacing the predetermined video picture with the example picture, edge smoothing and noise reduction processing are performed on the picture set after replacing the predetermined video picture with the example picture. The edge smoothing algorithm used in this embodiment may be, but is not limited to, the following algorithms: a mean filtering algorithm, a median filtering algorithm, a Gaussian filtering algorithm, and a bilateral filtering algorithm. The noise reduction processing method used in this embodiment may be, but is not limited to, the following methods: wavelet threshold denoising algorithm based on wavelet domain, median filtering algorithm based on spatial domain, and total variation image denoising algorithm.

According to the embodiment, the video picture whose similarity reaches the preset similarity threshold in the video is replaced by the high quality, high resolution example picture to improve the quality of the video picture, thereby improving the quality of the encoded video. Therefore, the video processing method according to the present embodiment solves the problem that the video quality is uneven and cannot fully satisfy the practical application.

According to an embodiment of the present disclosure, there is also provided a video processing apparatus which can be disposed in any terminal that can capture video. As shown in FIG. 2, the video processing apparatus may include a decoding module 201, a replacement module 202, and an encoding module 203 coupled in sequence. In this embodiment, the decoding module 201 is configured to decode the video to be processed to obtain a picture set composed of a plurality of video pictures; the replacement module 202 is configured to replace the predetermined video picture with an example picture, wherein the predetermined video picture is The picture set is similar to the sample picture with a similarity greater than the preset similarity threshold; the encoding module 203 is configured to encode the picture set after replacing the predetermined video picture with the example picture to obtain the processed video.

In the embodiment of the present disclosure, the video to be processed may be a normal video or a panoramic video that requires higher video quality. When the video to be processed is coded and processed by the decoding module 201 and the encoding module 203, the video to be processed can be processed using various methods of existing lossless compression or lossy compression. Here, the codec techniques used include, but are not limited to, the following techniques: predictive coding techniques, quantized coding techniques, Shannon coding techniques, and Huffman coding. In actual operation, an appropriate codec technology can be selected according to the video capturing device and the specific application of the video. It should be noted that the method of decoding and encoding the video should be consistent. For example, in the case of decoding a video segment by Huffman coding techniques, it is also necessary to encode the video picture using Huffman coding techniques after being replaced with the example picture.

Before replacing the predetermined video picture with the example picture, it is necessary to take a picture of the video to be processed in the same scene as an example picture. In order to achieve better results, try to take a sample picture at the same angle as the video to be processed, and try to take the corresponding sample picture where the scene changes greatly. At the same time, taking high-quality, high-resolution pictures as an example picture as much as possible can make the processed video quality better.

After the decoding module 201 decodes the processed video and obtains the image set, it is detected by the detecting unit whether the resolution of the predetermined video picture is smaller than the resolution of the example picture. In the case where the resolution of the predetermined video picture is smaller than the resolution of the example picture, the replacement module 202 replaces the predetermined video picture with the example picture. In an exemplary embodiment, the example picture is segmented into example picture blocks of a predetermined size by a segmentation unit while the predetermined video picture is also segmented into video picture blocks of a predetermined size. The size of each of the divided picture blocks is the same as the size of each of the divided video picture blocks. The similarity between the example picture block and the video picture block is calculated by the calculation unit. In the case where the similarity is greater than the preset similarity threshold, the replacement module 202 replaces the video picture block with the example picture block. It should be understood that the calculation unit may perform the similarity calculation between the picture blocks using the following methods, but is not limited to the following methods: a histogram matching method, a matrix decomposition method, and an image similarity calculation method based on feature points.

In order to improve the quality of the processed video to achieve a better display effect, the above apparatus further includes a processing module configured to replace the predetermined video picture with the example picture after replacing the predetermined video picture with the example picture The picture set performs edge smoothing processing and noise reduction processing. The edge smoothing algorithm used by the processing module may be, but is not limited to, the following algorithms: a mean filtering algorithm, a median filtering algorithm, a Gaussian filtering algorithm, and a bilateral filtering algorithm. The noise reduction processing method used by the processing module may be, but is not limited to, the following methods: wavelet threshold denoising algorithm based on wavelet domain, median filtering algorithm based on spatial domain, and total variation image denoising algorithm.

According to the embodiment, the video picture whose similarity reaches the preset similarity threshold in the video is replaced by the high quality, high resolution example picture to improve the quality of the video picture, thereby improving the quality of the encoded video. Therefore, the video processing apparatus according to the present embodiment solves the problem that the video quality is uneven and cannot fully satisfy the practical application.

According to an embodiment of the present disclosure, there is provided a terminal that can process a video to obtain a video of higher quality video. The terminal in this embodiment may be a mobile terminal with a camera, such as a dual camera mobile phone, a tablet, etc., and may also be a professional video capturing device such as a camera, a camera, or a panoramic camera. The video to be processed may be a video taken by the terminal according to the present embodiment, or may be a video captured in advance. The example picture may be a picture taken at the same time as the video to be processed, or may be a high resolution example picture that is taken as much as possible or as similar as possible to the video to be processed. In an exemplary embodiment, for example, the terminal is a camera, and the video to be processed is a panoramic video captured by a camera, and the example picture is a high resolution non-panoramic picture taken simultaneously with the panoramic video.

The application scenario of the terminal according to this embodiment is described in detail below by taking a panoramic video as an example.

S1, taking a panoramic video with the camera, and taking a plurality of high-resolution non-panoramic pictures in the same scene as an example picture while shooting the panoramic video;

S2, decoding the panoramic video to obtain a picture set composed of multiple panoramic pictures;

S3, processing the panoramic picture obtained by the video decoding by using the high quality example picture in the same scenario (for example, replacing the panoramic picture obtained by the video decoding with the high quality example picture in the same scene), and improving the image of the panoramic picture quality;

S4, encoding the processed picture set to obtain a panoramic video.

In step S3, the processing of the first picture in the decoded panoramic picture is different from the other pictures in the decoded panoramic picture. In an exemplary embodiment, step S3 includes the following sub-steps:

S31, filling the first picture with an image interpolation algorithm, so that the resolution of the filled picture is raised to the standard of the sample picture;

S32. Divide the panoramic picture into a panoramic picture block of a predetermined size, and divide the example picture into example picture blocks of a predetermined size;

S33, each panoramic picture block is compared with each of the example picture blocks, and the panoramic picture block whose similarity is greater than the preset similarity threshold is replaced by the example picture block;

S34, smoothing an edge of the example picture block in which the panoramic picture block is replaced, and performing noise reduction processing on all the panoramic pictures.

For other panoramic pictures other than the first panoramic picture, only S32 to S34 of the above sub-steps may be performed while processing. When the current panoramic picture is processed, the current panoramic picture can be replaced by the already processed panoramic picture as an example picture to improve the fit between the pictures and the picture quality.

According to an embodiment of the present disclosure, there is also provided a computer readable storage medium having stored therein a computer executable program, wherein when the computer executable program is run on a processor, At least the following steps can be performed:

Decoding the processed video to obtain a set of pictures consisting of multiple video pictures;

Replacing the predetermined video picture with the example picture, wherein the predetermined video picture is a video picture in which the similarity of the picture set to the example picture is greater than the similarity threshold;

The picture set after replacing the predetermined video picture with the example picture is encoded to obtain a processed video.

While the preferred embodiment of the present disclosure has been disclosed for purposes of illustration, those skilled in the art will recognize that various modifications, additions and substitutions are possible, and the scope of the present disclosure should not be limited to the embodiments described above.

Claims

A video processing method includes the following steps:

Decoding the processed video to obtain a set of pictures consisting of multiple video pictures;

Replacing the predetermined video picture with an example picture, wherein the predetermined video picture is a video picture in which the similarity of the picture set to the example picture is greater than a preset similarity threshold;

The set of pictures after replacing the predetermined video picture with the example picture is encoded to obtain a processed video.
The video processing method according to claim 1, wherein the step of replacing the predetermined video picture with the example picture comprises:

Detecting whether a resolution of the predetermined video picture is smaller than a resolution of the example picture;

In a case where the resolution of the predetermined video picture is smaller than the resolution of the example picture, the predetermined video picture is replaced with the example picture.
The video processing method according to claim 1, wherein the step of replacing the predetermined video picture with the example picture comprises:

Dividing the example picture into an example picture block of a predetermined size while segmenting the predetermined video picture into the video picture block of the predetermined size;

Calculating a similarity between the example picture block and the video picture block;

In the case that the similarity is greater than the preset similarity threshold, the video picture block is replaced with the example picture block.
The video processing method according to any one of claims 1 to 3, wherein after replacing the predetermined video picture with the example picture, the method further comprises:

Edge smoothing processing and noise reduction processing are performed on the picture set after replacing the predetermined video picture with the example picture.
The video processing method according to any one of claims 1 to 3, wherein before the decoding of the video to be processed to obtain a picture set consisting of a plurality of video pictures, the method further comprises:

A picture in the same scene as the to-be-processed video is taken as an example picture.
A video processing device comprising:

a decoding module configured to decode a video to be processed to obtain a picture set composed of a plurality of video pictures;

a replacement module, configured to replace the predetermined video picture with an example picture, wherein the predetermined video picture is a video picture in which the similarity of the picture set to the example picture is greater than a preset similarity threshold;

An encoding module configured to encode the set of pictures after replacing the predetermined video picture with the example picture to obtain a processed video.
The video processing device of claim 1, wherein the replacement module comprises: a detecting unit configured to detect whether a resolution of the predetermined video picture is smaller than a resolution of the example picture;

The replacement module is further configured to replace the predetermined video picture with the example picture if a resolution of the predetermined video picture is less than a resolution of the example picture.
The video processing device according to claim 1, wherein the replacement module comprises: a dividing unit configured to divide the example picture into example picture blocks of a predetermined size, and divide the predetermined video picture into the a video picture block of a predetermined size; and a computing unit configured to calculate a similarity between the example picture block and the video picture block, and

The replacement module is further configured to replace the video picture block with the example picture block if the similarity is greater than the preset similarity threshold.
The video processing device according to any one of claims 6 to 8, further comprising:

And a processing module configured to perform edge smoothing processing and noise reduction processing on the set of pictures after replacing the predetermined video picture with the example picture.
A terminal comprising the video processing device of any one of claims 6 to 9.
A computer readable storage medium having stored therein a computer executable program, wherein when the computer executable program is run on a processor, performing any one of claims 1-5 Said method.