CN107027029B - High-performance video coding improvement method based on frame rate conversion - Google Patents
High-performance video coding improvement method based on frame rate conversion Download PDFInfo
- Publication number
- CN107027029B CN107027029B CN201710117876.8A CN201710117876A CN107027029B CN 107027029 B CN107027029 B CN 107027029B CN 201710117876 A CN201710117876 A CN 201710117876A CN 107027029 B CN107027029 B CN 107027029B
- Authority
- CN
- China
- Prior art keywords
- frame
- block
- intra
- motion vector
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
- H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a high-performance video coding method combined with frame rate conversion aiming at the HEVC coding standard. The method mainly comprises the steps of carrying out self-adaptive frame extraction on an original video at an encoding end to reduce the transmission data volume; the low frame rate video carries out motion vector estimation on a lost frame by HEVC coding and combining a reference frame motion vector extracted from a code stream and intra-frame block information, and the lost frame is reconstructed by combining the motion vector at a decoding end and an improved bidirectional motion compensation inserted frame based on block coverage. The invention reduces the data amount of transmission and saves the transmission bandwidth due to the self-adaptive frame extraction; the process of motion estimation for the decoded video is saved, so that a large amount of time is saved; the improved block coverage based bi-directional motion compensated frame interpolation guarantees the quality of the reconstructed video. The experimental result shows that compared with HEVC standard coding, the algorithm saves the code rate by about 15% and saves the coding time by about 40% on the premise of ensuring that the PSNR of the reconstructed video is close to the consistency at the medium and low code rate end.
Description
Technical Field
The invention relates to the problem of reducing video coding rate in the field of image communication, in particular to a method for reducing coding rate by combining High Efficiency Video Coding (HEVC) standard with video frame rate conversion.
Background
With the popularization and development of high definition digital television (HDTV) and high-end multimedia systems, people have higher requirements for subjective enjoyment of videos, which puts higher requirements on the quality of video program sources, and the higher the quality of the program sources, the more the data volume of the videos, the larger the bandwidth required for transmission. Meanwhile, the video coding standard is continuously updated, and through the development and improvement of video coding standards such as MPEG, H.264/AVC and the like, a new generation of video coding standard HEVC has also been released. And compared with H.264/AVC, the HEVC coding standard saves nearly 50% of coding rate, but for high-definition video, the transmission data volume is still large. Therefore, under the limited bandwidth, how to reduce the transmission data amount and transmit the coded code stream in real time on the premise of ensuring the quality of the decoded video is still a hot point of research in the field of video coding.
Motion information is crucial to video reconstruction during video compression. In the frame rate conversion-based HEVC improved video compression coding framework, the acquisition of motion information and the post-processing of the motion information are improved. In a standard HEVC coding framework, no deep research is conducted on information recovery of a lost frame part, so that finding an efficient video frame insertion recovery method is of great significance for improving compression efficiency. At present, most of the methods widely used by academia for recovering lost frames are to perform motion estimation on decoded low frame rate video again to obtain motion vectors used in the frame interpolation process, and have a certain effect.
The frame rate conversion method is used as a technology for improving the compression rate, and is combined with the standard HEVC coding, so that the coding rate can be effectively reduced. Sung-he Lee et al propose to reduce the frame rate of the video at the encoding end to reduce the number of encoding frames, reduce the data of the transmission code stream, and then insert frames at the decoding end to restore the video at the original frame rate. Conventional frame rate up techniques are divided into two categories: non-motion compensated and motion compensated. In the non-motion compensation frame interpolation recovery process, motion information of an object in an original video is not considered, and pixel values of a frame to be interpolated are recovered only through pixel values of corresponding positions of two reference frames of a previous frame and a current frame, such as a frame repetition method and a frame averaging method; the motion compensation frame interpolation considers the motion of an object in a scene, and reconstructs a frame to be interpolated by combining motion vector information and pixel values in a reference frame, and the method can effectively reduce the jitter introduced by a frame repetition method and the blurring phenomenon introduced by a frame averaging method. Choi et al have proposed the two-way motion compensation interpolation (BMCI) suitable for the up-conversion technology of the frame rate specially, the source of the motion vector information used is different according to inserting the frame, the motion compensation frame can be divided into two kinds, one kind is based on compressing the field, this method is through extracting the motion vector information of the adjacent frame from the code stream after the code, used for the motion compensation frame, in this way, has saved and moved the process estimated in order to obtain the motion vector again to the video after the code, have greatly reduced calculated amount and time; the second method is based on the pixel domain, i.e. motion estimation is performed again on the decoded video, and then the obtained motion vector information is used to perform frame interpolation reconstruction. Although the effect of the motion compensation frame interpolation method is obviously better than that of the non-motion compensation method, the video reconstructed by the conventional motion compensation frame interpolation method has holes and block effects.
Disclosure of Invention
In order to solve the problems of holes and blocking effects existing in reconstructed frames in the traditional frame rate increasing process and improve the performance of video compression coding and the quality of reconstructed videos, the invention provides a high-performance video compression coding improvement method combined with frame rate conversion, which reduces the video coding rate on the premise of ensuring the video quality and has better application prospect in a real-time transmission system with limited bandwidth.
The basic idea of the invention is to perform self-adaptive frame extraction on an original video at a coding end, perform coding and decoding on a low frame rate video, extract motion vector information and special intra-frame block information in the HEVC coding process from code stream information, and perform frame rate up-sampling on the decoded video by combining motion vectors at a decoding end to restore the original frame rate, thereby achieving the purposes of reducing the number of coding frames and reducing the coding rate.
The invention provides an improved HEVC video compression coding framework combined with frame rate conversion aiming at an HEVC video coding standard. The method mainly comprises the steps of carrying out HEVC standard video coding on a low-frame-rate video subjected to self-adaptive frame extraction, and extracting related motion vector information and corresponding intra-frame block information from transmitted code stream information, wherein the motion vector information is stored in the code stream information in a PU block mode due to a special block division mode of HEVC, namely pixel positions in the same PU block have the same motion vector, under an IPPP inter-frame configuration coding mode, an encoder can simultaneously carry out intra-frame prediction and inter-frame prediction to obtain residual information, although the prediction modes of most coding blocks are inter-frame prediction, when the residual information obtained by the inter-frame prediction is not accurate enough, the encoder uses the residual obtained by automatically using the intra-frame prediction for coding, so if the block information extracted from the code stream information corresponds to an intra-frame coding block, the motion vector is 0, and the void or block effect can be caused if the frame is inserted, therefore, before the frame interpolation is recovered, the motion information of the intra blocks is estimated to ensure that each coding block in the frame has corresponding motion vector information; and performing smooth filtering processing on the obtained motion vector, and restoring the original frame rate by performing bidirectional motion compensation frame interpolation based on block coverage on the decoded video by combining the processed motion vector information. The method mainly comprises the following steps:
(1) before coding, carrying out self-adaptive frame extraction on an original frame rate video, and setting the maximum frame extraction number between two adjacent frames to be 2 and the minimum frame extraction number to be 0;
(2) coding the video after frame extraction, transmitting the video to a decoding end, and extracting related motion vector information and special intra-frame block information in HEVC from transmitted code stream information;
(3) performing bidirectional motion estimation on the lost frame through the motion vector information obtained in the step (2) to obtain an initial motion vector;
(4) the position of an intra-frame coding block in a frame is determined by combining intra-frame block information acquired from code stream information, and because the motion information of the intra-frame coding block cannot be used for frame interpolation reconstruction, the motion vector information of the intra-frame block is replaced by the weighted average value of the motion vectors of the adjacent blocks of the intra-frame block and the blocks at the same position in two adjacent reference frames, so that the corrected motion vector of the lost frame is obtained;
(5) by motion estimation of motion vectors of the intra blocks, each coding block has own motion vector information, and motion vector smoothing processing is carried out on each 4 x4 block;
(6) combining the final motion vector information of the lost frame obtained in the previous step, and performing interpolation frame reconstruction on the low frame rate decoding video by a bidirectional motion compensation interpolation frame method based on block coverage;
in the above technical solution of the present invention, the adaptive frame extraction method determines that two adjacent reference frames can be discarded continuously at most, the minimum frame loss is 0, and specifically determines whether a frame is a discardable frame according to the following criteria:
wherein f isi-1(i, j) represents the pixel value of the previous frame at position (i, j), fi(i, j) represents the pixel value of the current frame at position (i, j), m and n represent the width and height of the video, respectively, MSADiIt represents the average value of the absolute error sum of the pixel values of the corresponding positions of two adjacent frames, and R represents the relative change rate of the adjacent frames. Setting the threshold of the maximum continuous frame extraction number between two adjacent frames as 2, the MSAD threshold as 35 and the R threshold as 3.
In the above technical solution of the present invention, a conventional bidirectional motion compensation frame interpolation calculation method is:
wherein Ft(x,y) represents the reconstructed value of the frame to be interpolated at position (x, y), Ft-1And Ft+1The sub-table represents the previous reference frame and the current reference frame, (V)x,Vy) For the motion vector between the front reference frame and the back reference frame of the frame to be inserted obtained in the code stream,
it represents the initial motion vector of the frame to be interpolated relative to the previous and next reference frames calculated from the reference frame motion vector information.
In the above technical solution of the present invention, a specific method for correcting a motion vector of an intra-coded block by combining intra-block information is as follows:
wherein, MVintraRepresenting intra block motion information estimate, MVprevIndicating the motion vector, MV, at the same location of the previous reference frame as the intra blockcurrIndicating the motion vector, MV, in the subsequent reference frame at the same location as the block in the frameiRepresenting block motion vectors, w, adjacent to the intra blockiRepresents the compensation weight, n is the number of adjacent blocks, and for the compensation weight wiThe setting rule of (1) is: if the size of the adjacent block is the same as that of the intra block, the weight is 1, otherwise, the difference is determined according to the difference value of the division depths of the adjacent block and the intra block in the HEVC, the weight is 0.5 if the difference is 1, the difference is 2, the weight is 0.25, and so on. If there is an intra block in the neighboring blocks, the intra block motion vector is set to 0 if all the surrounding blocks are intra blocks.
In the above technical solution of the present invention, the method for performing smooth filtering on the modified motion vector includes:
in the above technical solution of the present invention, the method for reconstructing the interpolated frame of the low frame rate decoded video by the bidirectional motion compensation frame interpolation method based on block covering comprises:
wherein F (x, y) represents the block reconstruction value of the frame to be interpolated at the position (x, y), n represents the number of the reference blocks covered by the block to be interpolated,
representing the motion vector of the reference block covered by the block to be inserted.
The method according to the present invention can be used to construct an HEVC video codec for implementing the frame rate conversion-based high performance video coding improvement method.
The invention is completed based on the following idea analysis:
the frame rate conversion technology as an effective means for improving the video coding efficiency can be combined with a standard video coding method, and the specific method for self-adaptive frame extraction in the invention comprises the following steps: the criterion for determining whether a frame is a discardable frame is the mean absolute error (MSAD) of two framesi) And relative rate of change (R) when MSADiIf the value of R is greater than the given threshold value, the ith frame is judged to be a scene jump frame, and the frame is reserved without frame extraction; on the contrary, the frame is regarded as an extractable frame, after one frame is judged, whether the next adjacent frame is an extractable frame or not is continuously judged, and therefore the problem exists. Through experimental comparison, when the threshold of the maximum continuous frame extraction number between two adjacent frames is set to be 2 frames, the MSAD threshold is 35, and the R threshold is 3, the effect is better.
The low frame rate video after the self-adaptive frame extraction is coded through an HEVC (high efficiency video coding) frame, motion vector information is obtained from a coded code stream and is used for preliminarily estimating a motion vector of a lost frame, the initial motion vector is corrected and is used for bidirectional motion compensation frame insertion, and therefore the quality of the video after frame insertion recovery is improved.
In a special block division mode of HEVC, motion vector information is stored in code stream information in a PU block manner, that is, pixel positions in the same PU block have the same motion vector, in an IPPP inter-frame configuration coding mode, an encoder performs intra-frame prediction and inter-frame prediction at the same time to acquire residual information, and although the prediction modes of most coding blocks are inter-frame prediction, when the residual information acquired by inter-frame prediction is not accurate enough, the encoder uses the residual obtained by intra-frame prediction automatically for coding, so if block information extracted from the code stream information corresponds to an intra-frame coding block, the motion vector is 0, and if the block information is used for frame interpolation, a hole or block effect is caused, and therefore, before frame interpolation is performed, motion information of these intra-frame blocks is estimated. In the video, not only the time domain correlation exists between sequence frames, but also the spatial correlation exists in each frame, so the motion information of adjacent blocks of a block to be interpolated in the frame interpolation process can also be used as a reference, and for an intra-frame coding block, the motion vector of the intra-frame coding block is estimated by using the motion information of the spatial adjacent block of the block to be interpolated and a block at the same position in a reference frame.
After the estimation of the motion vector of the intra block, each coding block has its own motion information, before the interpolation recovery of each 4 × 4 block in the frame to be interpolated, the average value of the eight blocks adjacent to the block and its own motion vector is taken as the final motion vector in the frame recovery process, if the block to be interpolated is located in the boundary region, the motion vector of the reference block in the boundary is taken for reference, for example, if we need to interpolate the block at the bottom right corner of the image, the corresponding block is taken from the reference frame, and the left block, the top block and the three blocks at the top left are taken for reference. And interpolating boundary blocks at other corners, and so on.
The realization process of the interpolation frame reconstruction of the low frame rate decoding video by the bidirectional motion compensation interpolation frame method based on the block coverage is as follows: most information in HEVC, such as motion vectors, is stored and transmitted in different partitioned block formats and is reconstructed in a block pixel value reconstruction mode in a video reconstruction process, so that on the basis of bidirectional frame interpolation, correlation on a hollow region in a video frame is fully considered, a region of a block to be interpolated is interpolated, and pixel values in different regions are interpolated by different methods, namely one block to be interpolated in a lost frame covers a reference block containing different motion information.
Compared with the standard HEVC video coding method, the method firstly adopts a self-adaptive frame extraction method to process a to-be-coded sequence, then extracts relevant motion vector information from code stream information to estimate the motion vector of a lost frame, corrects the motion vector of the lost frame by combining intra-frame block information in the code stream, obtains the motion vector finally used for video reconstruction by performing smooth filtering on the motion vector of each 4 x4 reconstruction block, and finally performs video reconstruction by a bidirectional motion compensation frame insertion method for block coverage. The method of the invention can reduce the data amount of the code stream and save the transmission bandwidth on the premise of ensuring the quality of the reconstructed video in the middle and low code rate section, and greatly improves the coding time efficiency due to omitting the process of motion estimation again of the lost frame.
Drawings
Fig. 1 is a flow chart of the improved method for high-performance video coding based on frame rate conversion according to the present invention.
FIG. 2 is a diagram illustrating adaptive frame extraction according to the present invention.
FIG. 3 is a schematic diagram of a bi-directional motion compensation frame interpolation method according to the present invention.
Fig. 4 is a schematic diagram of a smoothing method for an initial motion vector according to the present invention.
Fig. 5 is a schematic diagram of motion compensated interpolation based on block covering in the present invention.
Fig. 6 to 9 are graphs comparing the rate-distortion curves of the algorithm of the present invention and the HM16.0 standard coding method, wherein fig. 6 to 9 are graphs comparing the rate-distortion curves of the sequences parylene, fourpeoples, ParkScene, peoplesnstreet at QP of 22, 27, 32, 37, 42, 44, 46, 48, 50, respectively.
Detailed Description
The present invention is further described in detail with reference to the following examples, which are intended to be illustrative only and are not to be construed as limiting the scope of the invention, which is to be construed as being limited thereby.
The frame rate conversion-based high-performance video improved coding algorithm has the following comparison process with the coding method of the HEVC standard test model HM 16.0:
1. opening a standard HM16.0 test model, setting a configuration file to be lowdelay _ P _ main, coding and decoding a standard video test sequence under the condition that quantization parameters are respectively 22, 27, 32, 37, 42, 46, 48 and 50, and recording a code rate and a reconstructed video PSNR (Peak Signal to noise ratio) during standard HEVC video coding;
2. the method of the present invention compares the code rate with reference software HM16.0 of the HEVC video coding standard in above 1 and the reconstructed video PSNR. Meanwhile, the programs of the invention and the standard method are opened, the same configuration file is set, the video sequence is acquired under the frame rate by a self-adaptive frame extraction method before encoding, the original frame rate video sequence is encoded and decoded by a standard algorithm, the low frame rate video acquired under the frame rate is encoded and decoded by the algorithm of the invention, and relevant motion information and intra-frame block information are extracted from code stream information. For three video coding performances: peak signal-to-noise ratio (PSNR), code rate, and coding time (where PSNR represents the objective quality of the reconstructed video, code rate represents the data transmission amount, and coding time represents the time efficiency of video coding processing) are compared and analyzed, and the difference of comparison performance is evaluated by the following two indexes:
wherein, △ code rate represents the percentage of the difference between the code rate of the algorithm and the standard algorithm relative to the standard code rate, i.e. the percentage of the reduced code rate, △ T represents the percentage of the difference between the time of the algorithm and the standard encoding and decoding relative to the time of the standard encoding
3. The coding object is a standard HEVC test video, and the name, resolution and frame rate of the HEVC test video are as follows: PartyScene (832x480, 50 frames/sec), FourPeople (1280x720, 60 frames/sec), ParkScene (1920x1080, 24 frames/sec), PeopleOnStreet (2560x1600, 30 frames/sec).
4. Encoding an original frame rate video sequence in an HEVC (high efficiency video coding) mode by utilizing an HM16.0 standard method;
5. the method is utilized to encode the low frame rate video of the same sequence, and extract the relevant motion vector information and intra-frame block information from the code stream;
6. the method of the invention is utilized to carry out frame rate up-sampling on the decoded low frame rate video;
7. respectively recording related experimental data such as code rate, PSNR (Peak Signal to noise ratio), time and the like in the encoding and decoding processes of the two methods; wherein the inventive method saves the code rate relative to the standard method in case of near agreement of the reconstructed video objective quality (PSNR) as in table 1, and the inventive comparison with the standard algorithm in terms of time efficiency is as in table 2. The statistical result shows that in the medium-low bit rate section, compared with the HEVC standard method, the method disclosed by the invention can save the bit rate by about 15% on the premise of ensuring that the PSNR of the reconstructed video is close to consistency; while also saving nearly 40% in terms of coding time efficiency.
TABLE 1 comparison of coding rates between the inventive algorithm and the HM16.0 Standard algorithm
TABLE 2 comparison of codec time efficiency between the algorithm of the present invention and the HM16.0 Standard algorithm
Claims (8)
1. A high-performance video coding improvement method based on frame rate conversion is mainly characterized by comprising the following process steps:
(1) before coding, carrying out self-adaptive frame extraction on an original frame rate video, and setting the maximum frame extraction number between two adjacent reference frames to be 2 and the minimum frame extraction number to be 0;
(2) coding the video after frame extraction, decoding by a decoding end, and extracting related motion vectors and special intra-frame block information in HEVC from transmitted code stream information;
(3) performing bidirectional motion estimation on the lost frame through the motion vector obtained in the step (2) to obtain an initial motion vector;
(4) determining the position of an intra-frame coding block in a frame by combining intra-frame block information acquired from code stream information, wherein the motion information of the intra-frame coding block cannot be used for frame interpolation reconstruction, so that the motion vector information of the intra-frame coding block is replaced by the weighted average value of the motion vectors of adjacent blocks of the intra-frame coding block and blocks at the same position in two adjacent reference frames to obtain a lost frame correction motion vector;
(5) by motion estimation of motion vectors of the intra blocks, each coding block has own motion vector information, and motion vector smoothing processing is carried out on each 4 x4 block;
(6) and combining the final motion vector information of the lost frame obtained in the previous step, and performing interpolation reconstruction on the low frame rate decoded video by a bidirectional motion compensation interpolation method based on block coverage.
2. The method as claimed in claim 1, wherein the step (1) of adaptively extracting frames from the original frame rate video, and the specific method for determining whether a frame is a discardable frame is as follows:
wherein f isi-1(i, j) represents the pixel value of the previous reference frame at position (i, j), fi(i, j) represents the pixel value of the current reference frame at position (i, j), m and n represent the width and height of the video, respectively, MSADiThen representing the average value of the absolute errors of the pixels at the corresponding positions of two adjacent frames, R representing the relative change rate of the adjacent frames, if MSADiIf the value of R is greater than the given threshold value, the ith frame is judged to be a scene jump frame, and the frame is reserved without frame extraction; and otherwise, the frame is regarded as an extractable frame, and after one frame is judged, whether the next adjacent frame is an extractable frame or not is continuously judged.
3. The method of claim 2 wherein the MSAD is set during adaptive frame decimationiThe threshold is 35, the R threshold is 3, the maximum continuous frame number of two adjacent reference frames is 2, and the minimum is 0.
4. The improved frame rate transform-based high performance video coding method as claimed in any of claims 1 to 3, wherein the motion estimation method for the lost frame is:
wherein (V)x,Vy) For the motion vector between the front reference frame and the back reference frame of the frame to be inserted obtained in the code stream,
it represents the initial motion vector of the frame to be interpolated relative to the previous and next reference frames calculated from the reference frame motion vector information.
5. The improved frame rate transform-based high performance video coding method as claimed in claim 4, wherein the method for filling the intra block motion vector in combination with the intra block information comprises:
MVintrarepresenting intra block motion information estimate, MVprevAnd MVcurrRespectively representing motion vector information, MV, at the same position as the intra block in the previous and subsequent reference framesiRepresenting block motion vectors, w, adjacent to the intra blockiRepresents the compensation weight, and n is the number of adjacent blocks.
6. The method as claimed in claim 5, wherein the compensation weight w isiThe setting rule of (1) is: if the size of the adjacent block is the same as that of the intra-frame block, the weight is 1, otherwise, the difference is determined according to the difference value of the division depths of the adjacent block and the intra-frame block in HEVC, the weight is 0.5 if the difference is 1, the difference is 2, the weight is 0.25, and the like; if there is an intra block in the neighboring blocks, the intra block motion vector is set to 0 if all the surrounding blocks are intra blocks.
7. The improved frame rate conversion based high performance video coding method of claim 6, wherein a smoothing filtering process is performed for each 4 x4 block of motion vector information before the lost frame is recovered.
8. The improved method of high-performance video coding based on frame rate conversion as claimed in claim 7, wherein the lost frame is recovered by an improved bidirectional motion compensation method based on block covering, the specific method is:
wherein F (x, y) representsBlock reconstruction value of frame to be interpolated at position (x, y), n represents number of reference blocks covered by block to be interpolated, Ft-1(x, y) denotes a frame of time preceding the frame to be interpolated, Ft+1(x, y) represents a frame at a time subsequent to the frame to be interpolated,
representing the block motion vector of the frame to be interpolated at position (x, y).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710117876.8A CN107027029B (en) | 2017-03-01 | 2017-03-01 | High-performance video coding improvement method based on frame rate conversion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710117876.8A CN107027029B (en) | 2017-03-01 | 2017-03-01 | High-performance video coding improvement method based on frame rate conversion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107027029A CN107027029A (en) | 2017-08-08 |
| CN107027029B true CN107027029B (en) | 2020-01-10 |
Family
ID=59525569
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710117876.8A Active CN107027029B (en) | 2017-03-01 | 2017-03-01 | High-performance video coding improvement method based on frame rate conversion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107027029B (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109429067B (en) * | 2017-08-31 | 2022-10-25 | 腾讯科技（深圳）有限公司 | Dynamic picture compression method and device, computer equipment and storage medium |
| CN107566798A (en) * | 2017-09-11 | 2018-01-09 | 北京大学 | A kind of system of data processing, method and device |
| CN109729358A (en) * | 2017-10-30 | 2019-05-07 | 成都凯天电子股份有限公司 | Airborne video-with-audio recording plays back ruggedized monitor |
| CN109905717A (en) * | 2017-12-11 | 2019-06-18 | 四川大学 | A kind of H.264/AVC Encoding Optimization based on Space-time domain down-sampling and reconstruction |
| CN112969072B (en) * | 2019-03-11 | 2022-08-26 | 杭州海康威视数字技术股份有限公司 | Encoding and decoding method, device and equipment |
| CN111726636A (en) * | 2019-03-18 | 2020-09-29 | 四川大学 | HEVC (high efficiency video coding) coding optimization method based on time domain downsampling and frame rate upconversion |
| CN111726614A (en) * | 2019-03-18 | 2020-09-29 | 四川大学 | HEVC (high efficiency video coding) optimization method based on spatial domain downsampling and deep learning reconstruction |
| CN111741304A (en) * | 2019-03-25 | 2020-10-02 | 四川大学 | Method for combining frame rate up-conversion and HEVC (high efficiency video coding) based on motion vector refinement |
| CN110460856B (en) * | 2019-09-03 | 2021-11-02 | 北京达佳互联信息技术有限公司 | Video encoding method, video encoding device, video encoding apparatus, and computer-readable storage medium |
| CN110827380B (en) * | 2019-09-19 | 2023-10-17 | 北京铂石空间科技有限公司 | Image rendering method and device, electronic equipment and computer readable medium |
| CN110996170B (en) * | 2019-12-10 | 2022-02-15 | Oppo广东移动通信有限公司 | Video file playing method and related equipment |
| CN111641835B (en) * | 2020-05-19 | 2023-06-02 | Oppo广东移动通信有限公司 | Video processing method, video processing device and electronic equipment |
| CN112135081B (en) * | 2020-09-11 | 2022-10-28 | Oppo广东移动通信有限公司 | Mode control method and device, frame insertion chip and electronic equipment |
| CN113411581B (en) * | 2021-06-28 | 2022-08-05 | 展讯通信（上海）有限公司 | Motion compensation method, system, storage medium and terminal for video sequence |
| CN113556582A (en) * | 2021-07-30 | 2021-10-26 | 海宁奕斯伟集成电路设计有限公司 | Video data processing method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101094398A (en) * | 2006-06-20 | 2007-12-26 | 中兴通讯股份有限公司 | Video conversion process method based on transform of time resolution |
| CN102404568A (en) * | 2011-08-18 | 2012-04-04 | 上海交通大学 | P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding |
| CN106454378A (en) * | 2016-09-07 | 2017-02-22 | 中山大学 | Frame rate up-conversion video coding method based on deformation movement model and system |
-
2017
- 2017-03-01 CN CN201710117876.8A patent/CN107027029B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101094398A (en) * | 2006-06-20 | 2007-12-26 | 中兴通讯股份有限公司 | Video conversion process method based on transform of time resolution |
| CN102404568A (en) * | 2011-08-18 | 2012-04-04 | 上海交通大学 | P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding |
| CN106454378A (en) * | 2016-09-07 | 2017-02-22 | 中山大学 | Frame rate up-conversion video coding method based on deformation movement model and system |
Non-Patent Citations (3)
| Title |
|---|
| Deformable Motion Model for Frame Rate Up-Conversion in Video Coding;S.Shen 等;《2016 IEEE Region 10 Conference》;20161125;全文 * |
| Frame Rate Up-Conversion with True Motion Estimation and Adaptive Motion Vector Refinement;Chong Xu 等;《2011 4th International Congress on Image and Signal Processing》;20111017;全文 * |
| New Frame Rate Up-Conversion Algorithms with Low Computational Complexity;Un Seob Kim等;《IEEE Transactions on Circuits and Systems for Video Technology》;20130816;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107027029A (en) | 2017-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107027029B (en) | High-performance video coding improvement method based on frame rate conversion | |
| Zhou et al. | Efficient motion vector interpolation for error concealment of H. 264/AVC | |
| US8325822B2 (en) | Method and apparatus for determining an encoding method based on a distortion value related to error concealment | |
| US6192080B1 (en) | Motion compensated digital video signal processing | |
| US6862372B2 (en) | System for and method of sharpness enhancement using coding information and local spatial features | |
| MX2007000810A (en) | Method and apparatus for encoder assisted-frame rate up conversion (ea-fruc) for video compression. | |
| JP2007503776A (en) | Method and apparatus for minimizing the number of reference images used for inter coding | |
| US20080137741A1 (en) | Video transcoding | |
| KR20060043051A (en) | Method for encoding and decoding video signal | |
| CN109905717A (en) | A kind of H.264/AVC Encoding Optimization based on Space-time domain down-sampling and reconstruction | |
| KR20040102211A (en) | System for and method of sharpness enhancement for coded digital video | |
| CN114913250A (en) | Video coding and decoding system and method based on semantic segmentation and Hungarian algorithm | |
| CN117750020A (en) | Method, system, equipment and storage medium for learning video coding | |
| Wige et al. | In-loop denoising of reference frames for lossless coding of noisy image sequences | |
| US20050013496A1 (en) | Video decoder locally uses motion-compensated interpolation to reconstruct macro-block skipped by encoder | |
| CN111741304A (en) | Method for combining frame rate up-conversion and HEVC (high efficiency video coding) based on motion vector refinement | |
| CN111726636A (en) | HEVC (high efficiency video coding) coding optimization method based on time domain downsampling and frame rate upconversion | |
| Glantz et al. | Video coding using global motion temporal filtering | |
| CN107409211A (en) | A kind of video coding-decoding method and device | |
| KR101220097B1 (en) | Multi-view distributed video codec and side information generation method on foreground segmentation | |
| CN115002482A (en) | End-to-end video compression method and system using structural preservation motion estimation | |
| Suzuki et al. | Block-based reduced resolution inter frame coding with template matching prediction | |
| Sekiguchi et al. | A low-cost video frame-rate up conversion using compressed-domain information | |
| US9363534B2 (en) | Method and device for processing a video sequence | |
| Li et al. | Rate-distortion-optimized content-adaptive coding for immersive networked experience of sports events |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |