CN101207707A - System and method for advancing frame frequency based on motion compensation - Google Patents
System and method for advancing frame frequency based on motion compensation Download PDFInfo
- Publication number
- CN101207707A CN101207707A CN 200710172515 CN200710172515A CN101207707A CN 101207707 A CN101207707 A CN 101207707A CN 200710172515 CN200710172515 CN 200710172515 CN 200710172515 A CN200710172515 A CN 200710172515A CN 101207707 A CN101207707 A CN 101207707A
- Authority
- CN
- China
- Prior art keywords
- frame
- motion vector
- motion
- macro block
- interleave
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a frame frequency promotion method and a system based on motion estimation and compensation. With the system, a forward motion vector and a backward motion vector of two adjacent video frames are respectively estimated through a bi-directional motion estimation unit; the size of a macro block during the compensation is selected and whether a video switch appears or not is judged according the relation of the motion vector before the vector smoothing and after the vector smoothing; the direction of the bi-directional motion vector is unified through a frame motion vector integrated unit to be inserted and whether a newly divided frame macro block to be checked is a general macro block or an invisible/visible motion macro block is judged; the suitable motion vector to the frame macro block to be checked is distributed through a frame motion vector post-processing unit to be inserted, and finally an inwards inserted frame is generated according to the compensation algorithm. With the invention, a video lens switch can be processed in time; a relatively perfect frame to be inserted is generated when the invisible/visible motion and picture frame background is moved inwards/outwards, and with the invention, the size of the macro block is selected automatically and the precision and the noise of the compensation are balanced.
Description
Technical field
The present invention relates to digital audio-video signal treatment system and method, relate in particular to a kind of liquid crystal display screen frame frequency lifting system and method based on estimation and compensation.
Background technology
LCD has obtained development and utilization widely as the flat panel display instrument.Its major defect is to produce motion blur phenomenons such as holder tail and ghost when video display at present.LCD produces the motion image distortion and fuzzy reason mainly contains 3 points: the one, and the liquid crystal display response time is slow; The 2nd, the retention time of liquid crystal display is long; The 3rd, human eye will carry out integration from the luminance signal of diverse location pixel along the displaying contents direction of motion, produce fuzzy.
The main two aspect reasons in the past of industry are started with and are improved the LCD device dynamic image quality: the one, and the response time of improving liquid crystal; The 2nd, reduce the LCDs retention time.Experimental results show that the method that improves response speed is initially effective, but when the liquid crystal response time less than after 8 milliseconds of field times, the improvement effect of motion blur (MotionBlur) has just been stagnated, arrive a bottleneck.The raising of simple response time can not be eliminated the motion image blooming, reduces the liquid crystal retention time so search out a kind of method, promptly by improving the method for LCD liquid crystal display screen refresh rate to 120Hz, reduces every frame retention time of liquid crystal.General video source is 60Hz, if directly it is inputed to the 120Hz panel, can produce the mistake that motion is quickened.And if each frame is repeated once to input to the 120Hz panel again, its display effect and 60Hz are suitable, the motion scintillation can occur sometimes even also.
As shown in Figure 1, be a kind ofly more effectively to reduce the LCD screen protection and hold time method based on the frame frequency lifting algorithm of motion compensation.Promptly from the picture basis of original 60fps frame (frame) or (field),, really dope middle frame (field), export to the panel of 120Hz again by estimation and compensation.
During frame (field) in the middle of prediction, block matching algorithm is a motion estimation algorithm the most widely at present.And the background motion that camera motion causes, motion is blocked/is appeared, the motion estimation information that motion noise informations such as non-rigid shape deformations all may lead to errors.For solving such problem, frame interpolation method that has occurred such as application number are that the Chinese invention patent application of 200610071860.X and application number are 200510110112.3 Chinese invention patent application.They block/appear problem for solving motion, all on the basis of adjacent two frames, have used the 3rd adjacent frame or the 4th two field picture to carry out reprocessing.They have increased the accuracy of estimation, but store a plurality of picture frames for the design that realizes towards ASIC, and resource consumption is bigger, and operation time is oversize.
In order to overcome the blocking effect of motion search, some algorithms copy video compression standard to adopt the motion estimation algorithm of multiple macro block mode, though increased the precision of motion search.But it brings the hardware estimator very large computational complexity, and when motion compensation, the cost of macro block mode decision-making has also increased the implementation complexity of hardware.
Therefore, how to provide a kind of liquid crystal display screen frame frequency lifting method and system based on estimation and compensation to generate the more satisfactory interleave for the treatment of when moving in and out with the picture frame background, become the technical problem that industry needs to be resolved hurrily to block in motion.
Summary of the invention
The object of the present invention is to provide a kind of frame frequency lifting system and method based on estimation and compensation, only need according to adjacent two frame of video, can Fast estimation go out motion vector and generate ideal to treat interleave, video lens switched make timely processing, and block in motion and to generate the more satisfactory interleave for the treatment of when moving in and out with the picture frame background.
Another object of the present invention is to select suitable macroblock size, balanced compensated precision and comfort noise in motion compensation stage automatically according to the content of picture frame.
For achieving the above object, a kind of frame frequency lifting system based on estimation and compensation generates the more satisfactory interleave for the treatment of when moving in and out with the picture frame background to block in motion, and this system comprises and connecting successively:
Color space converting unit is used to obtain the monochrome information of two adjacent images frame, to obtain previous moment luminance frame f
T-1With current time luminance frame f
T+1
The bi-directional motion estimation unit according to adjacent two luminance signal frames, obtains the motion vector MV of preliminary both direction by motion estimation algorithm
fAnd MV
b
The motion vector smooth unit, by the method for the motion vector of adjacent macroblocks relatively, the motion vector noise from two aspects of room and time correction motion estimation procedure;
Frame motion vector comprehensive unit to be inserted, the motion vector that obtains through level and smooth bi-directional motion estimation unit on average comprehensively on a direction, obtain the new motion vector for the treatment of each pixel of interleave, and treat interleave and repartition macro block, motion vector in the unified new macro block;
Treat interleave motion vector post-processing unit, treat in the interleave and block/appear that perhaps image background moves in and out the corresponding with it macro block zone of the no motion vector that causes and carries out reprocessing, to generate the motion vector of these macro blocks because of motion;
Motion compensation generates frame unit to be inserted, according to the motion vector for the treatment of interleave, obtains treating that interleave pursues the color and the monochrome information of pixel, and interleave inserts between former frame and the present frame finishes the frame frequency lifting function with treating of generating.
The above-mentioned frame frequency lifting system based on estimation and compensation, wherein: the motion estimation algorithm of this bi-directional motion estimation unit comprises the decision-making part to the macroblock size of frame of video camera lens switching processing part and judgement motion compensation stage.
The above-mentioned frame frequency lifting system based on estimation and compensation, wherein: this frame motion vector comprehensive unit to be inserted is partly treated interleave according to the decision-making of the macroblock size of above-mentioned compensated stage and is repartitioned macro block, and motion vector in the macro block is unified.
Above-mentioned frame frequency lifting system based on estimation and compensation, wherein: this bi-directional motion estimation unit further comprises decimation blocks under the image, the propulsion search module, the reverse search module, the smoothed motion vector module, frame motion vector generation module to be inserted, and frame type to be inserted is judged and motion compensation macroblock size judge module.Wherein, the output of decimation blocks is connected to propulsion search module and reverse search module respectively under the image, the output signal of propulsion search module is successively through obtaining motion vector output behind smoothed motion vector module and the frame motion vector generation module to be inserted, the output signal of reverse search module obtains motion vector output after via frame motion vector generation module to be inserted, and the output of propulsion search module and smoothed motion vector module also inputs to frame type to be inserted respectively and judges and motion compensation macroblock size judge module.
Above-mentioned frame frequency lifting system based on estimation and compensation, wherein: this frame motion vector comprehensive unit to be inserted further comprises the average module of motion vector that connects successively, treat interleave division module, macro block special pattern determination module, and macroblock motion vector is unified module.
The present invention provides a kind of frame frequency lifting method based on estimation and compensation in addition, is used to be created on previous moment frame Img
T-1With current time frame Img
T+1Between the Img of interpolation frame
T, it may further comprise the steps:
Step a, to two adjacent color image frame frame Img
T-1And Img
T+1Carry out the colour gamut conversion, obtain two adjacent luminance graph picture frame f
T-1And f
T+1
Step b, to luminance frame f
T-1And f
T+1Two luminance frame f that sample and obtain dwindling
T-1', f
T+1';
Step c, to luminance frame f
T-1', f
T+1', from forward direction and back, obtain the motion vector MV of both direction respectively to carrying out motion search
fAnd MV
b
Steps d, to luminance frame f
T+1' in macro block add up, according to frame f
T+1' in the relation before the smoothed motion vector, between afterwards the critical macroblock number, judge whether to occur video lens and switch, and the macroblock partitions size of definite motion compensation stage.
Above-mentioned frame frequency lifting method based on estimation and compensation, it further may further comprise the steps:
Step e, the level and smooth motion vector MV afterwards of basis
fAnd MV
b, generate frame motion vector MV to be inserted on the both direction passage respectively according to the interpolation algorithm of motion vector
f/ 2 and MV
b/ 2;
Step f, forward direction treated the motion vector MV of interleave
f/ 2 and back to the motion vector MV that treats interleave
b/ 2 is unified on a direction;
Step g, according to the motion compensation macroblock size, repartition and treat interleave f
TBe new macro block;
Interleave f is being treated in step h, judgement
TIn special macro block;
Step I, treating interleave f
TIn, the motion vector of the pixel in the non-special macro block is unified into same value;
Step j, treat interleave f
TIn special macro block, select suitable motion vector by the motion vector selection algorithm, make and treat interleave f
TIn all macro blocks all have motion vector corresponding with it;
Step k, with two adjacent color image frame frame Img
T-1And Img
T+1As the reference frame, treat interleave f by what obtain
TMotion vector MV
c, bring movement compensating algorithm into, generate color framing Img to be inserted
T, video lens is switched handle simultaneously.
The above-mentioned frame frequency lifting method based on estimation and compensation, wherein: this motion search comprises sweep forward path and the sweep backward path that carries out simultaneously.
Above-mentioned frame frequency lifting method based on estimation and compensation, wherein: above-mentioned sweep forward path uses method for fast searching, and above-mentioned sweep backward path uses local full search in the hunting zone.
Above-mentioned frame frequency lifting method based on estimation and compensation, wherein: above-mentioned to luminance frame f
T+1' in macro block to add up be luminance frame f in the sweep forward
T+1' in the statistics that macro block carried out because sweep forward uses method for fast searching, final two paths of data can arrive at next functional module simultaneously.
Compared with prior art, a kind of frame frequency lifting system and method for the present invention based on estimation and compensation, only need according to adjacent two frame of video, can Fast estimation go out motion vector and generate ideal to treat interleave, video lens switched make timely processing, and block in motion and to generate the more satisfactory interleave for the treatment of when moving in and out with the picture frame background.And the present invention can select suitable macroblock size, balanced compensated precision and comfort noise in motion compensation stage automatically according to the content of picture frame.
Description of drawings
A kind of frame frequency lifting system and method based on estimation and compensation of the present invention is provided by following embodiment and accompanying drawing.
Fig. 1 is traditional frame frequency lifting method schematic diagram;
Fig. 2 is the structural representation of a kind of liquid crystal display screen frame frequency lifting system based on estimation and compensation of the present invention;
Fig. 3 is the structural representation of the bi-directional motion estimation unit of a kind of liquid crystal display screen frame frequency lifting system based on estimation and compensation of the present invention;
Fig. 4 is the structural representation of the frame motion vector comprehensive unit to be inserted of a kind of liquid crystal display screen frame frequency lifting system based on estimation and compensation of the present invention;
Fig. 5 is the example that compensates based on the motion search of piece coupling of the present invention;
Fig. 6 is a forward and backward estimation schematic diagram of the present invention;
Fig. 7 (a)～Fig. 7 (c) is that the motion vector in of the present invention blocking/appear zone generates exemplary plot;
Fig. 8 is that the present invention generates the desirable distribution example figure that treats interleave when the picture frame background moves in and out.
Embodiment
Below will be described in further detail a kind of frame frequency lifting system and method for the present invention based on estimation and compensation.
As shown in Figure 2, the invention provides a kind of frame frequency lifting system based on estimation and compensation, it comprises continuous successively: color space converting unit 1, bi-directional motion estimation unit 2, motion vector smooth unit 3, frame motion vector comprehensive unit 4 to be inserted is treated interleave motion vector post-processing unit 5, and motion compensation generates frame unit 6 to be inserted.
This color space converting unit 1 is used to obtain the two adjacent images frame, and promptly the monochrome information of former frame image and current frame image is used for follow-up estimation calculating, obtains previous moment luminance frame f
T-1With current time luminance frame f
T+1A preferred embodiment is: convert the rgb signal of DVI mouth input to YUV signal, obtain monochrome information Y.
This bi-directional motion estimation unit 2, according to adjacent two luminance signal frames, obtain the motion vector of preliminary both direction by the peculair motion searching algorithm, it has comprised frame of video camera lens switching processing part and has judged the decision-making part of the macroblock size of motion compensation stage.
This motion vector smooth unit 3, the method for the motion vector by the macro block in the adjacent image frame relatively, the motion vector noise from two aspects of room and time correction motion estimation procedure.
This frame motion vector comprehensive unit 4 to be inserted, forward direction frame motion vector to be inserted that this bi-directional motion estimation unit 2 is obtained and back on average comprehensively on a direction, obtain the new motion vector for the treatment of each pixel of interleave to frame motion vector to be inserted.Subsequently, its motion compensation macroblock size of determining according to above-mentioned compensated stage decision-making part is treated interleave and is repartitioned macro block, and motion vector in the macro block is unified.
This treats interleave motion vector post-processing unit 5, treats in the interleave to block/appear because of motion, and perhaps image background moves in and out the corresponding with it macro block zone of the no motion vector that causes and carries out reprocessing, generates the motion vector of these macro blocks.
This motion compensation generates frame unit 6 to be inserted, link to each other with the memory cell of original adjacent two frame image informations of storage, according to the motion vector for the treatment of interleave, call movement compensating algorithm, obtain treating that interleave pursues the color and the monochrome information of pixel, the frame of video camera lens switch instances that bi-directional motion estimation unit 2 is judged is duplicated the former frame processing, and treat that to what produce the interleave background moves in and out phenomenon and given processing because of the camera motion model, treating of will generating subsequently finished the frame frequency lifting function between interleave insertion former frame and the present frame.
As shown in Figure 3, above-mentioned bi-directional motion estimation unit 2 further comprises: decimation blocks 20 under the image, propulsion search module 21, reverse search module 22, smoothed motion vector module 23, frame motion vector generation module 24 to be inserted, and frame type to be inserted is judged and motion compensation macroblock size judge module 25.
Decimation blocks 20 under this image, and it is to previous moment luminance frame f
T-1With current time luminance frame f
T+1Two luminance frame f that sample and obtain dwindling
T-1', f
T+1', be used for the quantity that the motion search module macro block is lacked in letter, to simplify amount of calculation.
This propulsion search module 21 obtains f with method for searching motion
T+1' frame is to f
T-1' motion vector MV
f'.
This reverse search module 22 obtains f with method for searching motion
T-1' frame is to f
T+1' motion vector MV
b'.
This smoothed motion vector module 23, its basic structure is identical with above-mentioned motion vector smooth unit 3, to the motion vector MV that initially obtains
f' and MV
b' carry out respectively smoothly and denoising.
This frame motion vector generation module 24 to be inserted, it is according to the motion vector MV after level and smooth
f' and MV
b', first up-sampling recovers two width of cloth motion vector MV of original image frame sign
fAnd MV
bGenerate the frame motion vector to be inserted of both direction then according to the interpolation algorithm of motion vector: forward direction T motion vector MV constantly
f/ 2, the back is to T motion vector MV constantly
b/ 2.
This frame type to be inserted judges and motion compensation macroblock size judge module 25 that its statistical property according to the motion vector that estimates judges that whether occurring video lens between adjacent two frames switches.Again according to through before the smoothed motion vector module, the size of the relation decision motion compensation macro block of afterwards motion vector statistical property.
The enhancement mode diamond search method that this propulsion search module 21 uses.What this reverse search module 22 used is local full search method.Before this smoothed motion vector module 23, f
T+1' critical module statistics be N
PreAfter the level and smooth module of vector, f
T+1' critical module statistics be N
PostWork as N
PreDuring greater than certain threshold value, putting copyflag is 1, and the video lens frame appears in expression to be switched, and directly jumps into motion compensation and generates frame unit 6 to be inserted, duplicates the former frame image as treating interleave.
Work as N
PreWhen being not more than threshold value, compare N
PreAnd N
PostWhether satisfy certain and concern f
1If satisfy f
1, thinking that picture frame to be inserted is more clear, it is consistent with motion search macroblock size bsize that motion compensation macroblock size MCsize is set.If N
PreAnd N
PostSatisfy and concern f
2, think that then picture frame noise to be inserted is too much, need smoothly, it is big that motion compensation macro block MCsize is set, and reduces comfort noise.
As shown in Figure 4, above-mentioned frame motion vector comprehensive unit 4 to be inserted further comprises: the average module 40 of motion vector, treat interleave division module 41, and macro block special pattern determination module 42, and macroblock motion vector is unified module 43.
The average module 40 of this motion vector, its forward direction is treated the motion vector MV of interleave
f/ 2 and back to the motion vector MV that treats interleave
b/ 2 is unified on a direction.
This treats interleave division module 41, and it is repartitioned and treat interleave according to the macroblock size that the motion compensation macroblock size judge module in the above-mentioned bi-directional motion estimation unit 2 is determined.
This macro block special pattern determination module 42, it judges that in treating interleave whether this macro block is because of no motion vector and its correspondence of blocking/appearing and causing in the motion in the T-1 moment and the T+1 moment; Perhaps because the disappearance of background and immigration also produce no motion vector and its correspondence.
This macroblock motion vector is unified module 43, and the motion vector of the pixel in its same macro block is unified into same vector value, compensates again.
In frame frequency lifting method, the invention provides a kind of previous moment frame Img that is used to be created on
T-1With current time frame Img
T+1Between the Img of interpolation frame
TMethod, may further comprise the steps:
Step 3, sweep forward path: with f
T+1' be reference frame, with f
T-1' be search frame, to f
T+1' in each macro block carry out propulsion search, obtain motion vector MV
f'.
Step 4, carry out the sweep backward path simultaneously: with f
T-1' be reference frame, with f
T+1' be search frame, to f
T-1' in each macro block carry out reverse search, obtain motion vector MV
b'.
Above-mentioned sweep forward path uses method for fast searching, such as improved diamond search method etc.Above-mentioned sweep backward path uses local full search in the hunting zone, prevent that sweep forward is absorbed in local smallest point, produces the mistake search.
Step 5, to luminance frame f in the sweep forward
T+1' in macro block add up, if the number of critical macro block reaches certain value, it is 1 that video content switching mark copyflag value is set.Critical macro block is meant that its motion vector of a certain macro block is (d
x, d
y), work as d
xPerhaps d
yAbsolute value when equaling the motion search range parameter p, we just claim that this macro block is critical macro block.
To forward motion vector MV
f' to carry out vector level and smooth, statistics frame f
T+1' at the relation before the smoothed motion vector, between afterwards the critical macroblock number, the macroblock size MCsize when deciding motion compensation.
Step 6, the level and smooth motion vector MV afterwards of basis
f' and MV
b', first up-sampling recovers two width of cloth motion vector MV of original image frame sign
fAnd MV
bGenerate frame motion vector MV to be inserted on the both direction passage respectively according to the interpolation algorithm of motion vector then
f/ 2 and MV
b/ 2.It should be noted that the interleave time point of demonstration example here is at the intermediate point of two frames constantly.And according to the difference of interleave time point, general frame motion vector to be inserted is MV
f/ k and MV
b/ k.
Step 7, forward direction treated the motion vector MV of interleave
f/ 2 and back to the motion vector MV that treats interleave
b/ 2 is unified on a direction.Such as unified forward direction frame motion vector to be inserted be
Generate the desirable distribution example figure that treats interleave when promptly picture frame background as shown in Figure 8 moves in and out.
Step 8, according to the macroblock size MCsize that step 5 is determined, repartition and treat interleave f
TBe new macro block.
Interleave f is being treated in step 9, judgement
TIn, which macro block is special macro block, promptly because motion is blocked/appeared or the disappearance of background and immigration and cause not having the macro block of motion vector and its correspondence, promptly the motion vector that blocks/appear the zone as shown in Figure 7 generates exemplary plot.
Step 10, treating interleave f
TIn, the motion vector of the pixel in the non-special macro block is unified into same value.
Step 11, treat interleave f
TIn special macro block, according to the principle of motion vector spatial continuity,, select suitable motion vector by selection algorithm the motion vector of the existence in these macro block eight neighborhood macro blocks, compose and give this macro block.Make and treat interleave f
TIn all macro blocks all have motion vector corresponding with it.
Step 12, with two adjacent color image frame frame Img
T-1And Img
T+1As the reference frame, treat interleave f by what obtain
TMotion vector MV
c, bring movement compensating algorithm into, generate color framing Img to be inserted
TWhen video content switching mark copyflag is 1, directly with previous image frame Img
T-1Duplicate a frame, as the picture frame Img that inserts
T, do not carry out the movement compensating algorithm of step 6 to step 11.
As shown in Figure 5, it is the exemplary plot based on the motion search compensation of piece coupling, promptly from original adjacent image frame, search out the motion vector MV of each macro block, estimate the motion vector for the treatment of each macro block correspondence in the interleave piece according to method provided by the invention again, at frame frequency is that 60 hertz of examples that are promoted to 120 hertz require down, and the motion vector of the corresponding macro block of interleave is MV/2.If (x
T-1, y
T-1) and (x
T+1, y
T+1) corresponding the same point in space, the forward motion vector MV of initial ranging so
f=(x
T-1-x
T+1, y
T-1-y
T+1), then to motion vector MV
b=(x
T+1-x
T-1, y
T+1-y
T-1).
As shown in Figure 6, it is a forward and backward estimation schematic diagram.Because relations such as macroblock partitions, in fact MV
fAnd MV
bThe like that just opposite in sign shown in the image pattern not, and absolute value is also different.When the frame frequency that promotes is unrestricted, if adjacent t
1And t
2Between t
iConstantly need treat interleave, then the forward direction coefficient
The back is to coefficient
Treating interleave motion vector comprehensive unit, unification to the comprehensive interleave motion vector of forward direction is
As shown in Figure 7, from former frame f
T-1To present frame f
T+1An object is arranged from the A macro block C macro block that moves right, f
TIt is the constantly desirable correct interpolation frame of T.For the reverse search, the A macro block is to appear macro block, and the C macro block is to block macro block.And for sweep forward, then the C macro block is to appear macro block, and the A macro block is to block macro block.A macro block to search is an example later on, and shown in Fig. 7 (a), it is f
T-1Reverse estimated vector figure.Shown in Fig. 7 (b), its generation treat interleave f
TMotion vector figure, suppose that the A macro block is a static background, can see that it is as the macro block that newly appears, in treating interleave, there are not motion vector and its correspondence, usage space of the present invention is faced the motion vector of territory piece, for example D macro block and E macro block are handled it, obtain the motion vector (present embodiment is zero) of A macro block, shown in Fig. 7 (c).For occlusion area C macro block, in sweep backward, perhaps can compose wrong motion vector, but in sweep forward, it be revised according to preceding method.
As shown in Figure 8, suppose from former frame f
T-1To present frame f
T+1, background is moved from right to left, so for sweep backward, f
T-1Left field can shift out in the frame.And the right side has the zone immigration, and these all can produce the motion vector mistake, does not perhaps have the situation of motion vector and this zone correspondence.Backward motion vector MV for example
bObtain wrong motion vector at a range searching; The back macro block district that has no motion vector correspondence to frame motion vector to be inserted by its generation.The propulsion search is similar.Through special macro block post-processing unit, all there is corresponding motion vector in all macro blocks.The movement compensating algorithm of corresponding improvement is as follows:
The sending area:
The receiving area:
In sum, the present invention is by the bi-directional motion search passage, and follow-up frame motion vector to be inserted is comprehensive and the motion vector post-processing algorithm, to moving in and out the corresponding and corresponding macro block of the erroneous motion vectors zone of no motion vector that motion produces because motion is blocked/appeared with background in the motion compensation, carry out effective processing and correction, simultaneously comfort noise has been carried out the self adaptation correction.And the present invention only needs according to adjacent two frame of video, but Fast estimation go out motion vector and generate ideal to treat interleave, video lens is switched makes timely processing, and block in motion and to generate the more satisfactory interleave for the treatment of when moving in and out with the picture frame background.
Claims (10)
1. frame frequency lifting system based on estimation and compensation generates the more satisfactory interleave for the treatment of when moving in and out with the picture frame background to block in motion/to appear, it is characterized in that, comprises connecting successively:
Color space converting unit is used to obtain the monochrome information of two adjacent images frame, to obtain previous moment luminance frame f
T-1With current time luminance frame f
T+1
The bi-directional motion estimation unit according to adjacent two luminance signal frames, obtains the motion vector MV of preliminary both direction by motion estimation algorithm
fAnd MV
b
The motion vector smooth unit, by the method for the motion vector of adjacent macroblocks relatively, the motion vector noise from two aspects of room and time correction motion estimation procedure;
Frame motion vector comprehensive unit to be inserted, the motion vector that obtains through level and smooth bi-directional motion estimation unit on average comprehensively on a direction, obtain the new motion vector for the treatment of each pixel of interleave, and treat interleave and repartition macro block, motion vector in the unified new macro block;
Treat interleave motion vector post-processing unit, treat in the interleave and block/appear that perhaps image background moves in and out the corresponding with it macro block zone of the no motion vector that causes and carries out reprocessing, to generate the motion vector of these macro blocks because of motion;
Motion compensation generates frame unit to be inserted, according to the motion vector for the treatment of interleave, obtains treating that interleave pursues the color and the monochrome information of pixel, and interleave inserts between former frame and the present frame finishes the frame frequency lifting function with treating of generating.
2. the frame frequency lifting system based on estimation and compensation as claimed in claim 1 is characterized in that: the motion estimation algorithm of this bi-directional motion estimation unit comprises the decision-making part to the macroblock size of frame of video camera lens switching processing part and judgement motion compensation stage.
3. the frame frequency lifting system based on estimation and compensation as claimed in claim 2, it is characterized in that: this frame motion vector comprehensive unit to be inserted is partly treated interleave according to the decision-making of the macroblock size of above-mentioned compensated stage and is repartitioned macro block, and motion vector in the macro block is unified.
4. the frame frequency lifting system based on estimation and compensation as claimed in claim 1, it is characterized in that: this bi-directional motion estimation unit further comprises decimation blocks under the image, the propulsion search module, the reverse search module, the smoothed motion vector module, frame motion vector generation module to be inserted, and frame type to be inserted is judged and motion compensation macroblock size judge module, wherein, the output of decimation blocks is connected to propulsion search module and reverse search module respectively under the image, the output signal of propulsion search module is successively through obtaining motion vector output behind smoothed motion vector module and the frame motion vector generation module to be inserted, the output signal of reverse search module obtains motion vector output after via frame motion vector generation module to be inserted, and the output of propulsion search module and smoothed motion vector module also inputs to frame type to be inserted respectively and judges and motion compensation macroblock size judge module.
5. the frame frequency lifting system based on estimation and compensation as claimed in claim 1, it is characterized in that: this frame motion vector comprehensive unit to be inserted further comprises the average module of motion vector that connects successively, treat interleave division module, macro block special pattern determination module, and macroblock motion vector is unified module.
6. the frame frequency lifting method based on estimation and compensation is used to be created on previous moment frame Img
T-1With current time frame Img
T+1Between the Img of interpolation frame
T, it is characterized in that may further comprise the steps:
To two adjacent color image frame frame Img
T-1And Img
T+1Carry out the colour gamut conversion, obtain two adjacent luminance graph picture frame f
T-1And f
T+1
To luminance frame f
T-1And f
T+1Two luminance frame f that sample and obtain dwindling
T-1', f
T+1';
To luminance frame f
T-1', f
T+1', from forward direction and back, obtain the motion vector MV of both direction respectively to carrying out motion search
fAnd MV
b
To luminance frame f
T+1' in macro block add up, according to frame f
T+1' in the relation before the smoothed motion vector, between afterwards the critical macroblock number, judge whether to occur video lens and switch, and the macroblock partitions size of definite motion compensation stage.
7. the liquid crystal display screen frame frequency lifting method based on estimation and compensation as claimed in claim 6 is characterized in that further may further comprise the steps:
According to the motion vector MV after level and smooth
fAnd MV
b, generate frame motion vector MV to be inserted on the both direction passage respectively according to the interpolation algorithm of motion vector
f/ 2 and MV
b/ 2;
Forward direction is treated the motion vector MV of interleave
f/ 2 and back to the motion vector MV that treats interleave
b/ 2 is unified on a direction;
According to the motion compensation macroblock size, repartition and treat interleave f
TBe new macro block;
Interleave f is being treated in judgement
TIn special macro block;
Treating interleave f
TIn, the motion vector of the pixel in the non-special macro block is unified into same value;
Treat interleave f
TIn special macro block, select suitable motion vector by the motion vector selection algorithm, make and treat interleave f
TIn all macro blocks all have motion vector corresponding with it;
With two adjacent color image frame frame Img
T-1And Img
T+1As the reference frame, treat interleave f by what obtain
TMotion vector MV
c, bring movement compensating algorithm into, generate color framing Img to be inserted
T, video lens is switched handle simultaneously.
8. the frame frequency lifting method based on estimation and compensation as claimed in claim 6 is characterized in that: this motion search comprises sweep forward path and the sweep backward path that carries out simultaneously.
9. the frame frequency lifting method based on estimation and compensation as claimed in claim 8 is characterized in that: above-mentioned sweep forward path uses method for fast searching, and above-mentioned sweep backward path uses local full search in the hunting zone.
10. the frame frequency lifting method based on estimation and compensation as claimed in claim 8 is characterized in that: above-mentioned to luminance frame f
T+1' in macro block to add up be luminance frame f in the sweep forward
T+1' in the statistics that macro block carried out because this search channels is used method for fast searching, final two paths of data can arrive at next functional module simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200710172515 CN101207707A (en) | 2007-12-18 | 2007-12-18 | System and method for advancing frame frequency based on motion compensation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200710172515 CN101207707A (en) | 2007-12-18 | 2007-12-18 | System and method for advancing frame frequency based on motion compensation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101207707A true CN101207707A (en) | 2008-06-25 |
Family
ID=39567545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200710172515 Pending CN101207707A (en) | 2007-12-18 | 2007-12-18 | System and method for advancing frame frequency based on motion compensation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101207707A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101621693B (en) * | 2009-07-31 | 2011-01-05 | 重庆大学 | Frame frequency lifting method for combining target partition and irregular block compensation |
| CN102123283A (en) * | 2011-03-11 | 2011-07-13 | 杭州海康威视软件有限公司 | Interpolated frame acquisition method and device in video frame rate conversion |
| CN102263925A (en) * | 2010-05-31 | 2011-11-30 | 郑朝钟 | Occlusion processing method |
| CN102447902A (en) * | 2011-09-30 | 2012-05-09 | 曾志华 | Method for selecting reference field and acquiring time-domain motion vector |
| CN101753795B (en) * | 2008-12-16 | 2012-05-16 | 晨星软件研发(深圳)有限公司 | Image processing method for determining motion vector of interpolation image and related device |
| CN102523419A (en) * | 2011-12-31 | 2012-06-27 | 上海大学 | Digital video signal conversion method based on motion compensation |
| CN102595089A (en) * | 2011-12-29 | 2012-07-18 | 香港应用科技研究院有限公司 | Frame-rate conversion using mixed bidirectional motion vector for reducing corona influence |
| CN101924950B (en) * | 2009-06-17 | 2012-08-29 | 普诚科技股份有限公司 | Data compression method for intelligent accelerating image display and device |
| CN102111613B (en) * | 2009-12-28 | 2012-11-28 | 中国移动通信集团公司 | Image processing method and device |
| CN102044207B (en) * | 2009-10-26 | 2013-02-06 | 群康科技(深圳)有限公司 | Circuit for adjusting setting time and holding time of driving chip |
| CN105100807A (en) * | 2015-08-28 | 2015-11-25 | 山东大学 | Motion vector post-processing based frame rate up-conversion method |
| CN110290360A (en) * | 2019-08-01 | 2019-09-27 | 浙江开奇科技有限公司 | Image splicing method and terminal device for panoramic video image |
| CN110933496A (en) * | 2019-12-10 | 2020-03-27 | Oppo广东移动通信有限公司 | Image data frame insertion processing method and device, electronic equipment and storage medium |
| CN111327926A (en) * | 2020-02-12 | 2020-06-23 | 北京百度网讯科技有限公司 | Video frame insertion method and device, electronic equipment and storage medium |
| CN111614911A (en) * | 2020-05-08 | 2020-09-01 | Oppo广东移动通信有限公司 | Image generation method and device, electronic equipment and storage medium |
| CN111641829A (en) * | 2020-05-16 | 2020-09-08 | Oppo广东移动通信有限公司 | Video processing method, device, system, storage medium and electronic equipment |
| CN114740965A (en) * | 2022-05-05 | 2022-07-12 | Oppo广东移动通信有限公司 | Processing method and device for reducing power consumption of terminal, terminal and readable storage medium |
| CN114898577A (en) * | 2022-07-13 | 2022-08-12 | 环球数科集团有限公司 | Road intelligent management system and method for peak period access management |
| WO2022267957A1 (en) * | 2021-06-21 | 2022-12-29 | 华为技术有限公司 | Video frame interpolation method and apparatus, and device |
-
2007
- 2007-12-18 CN CN 200710172515 patent/CN101207707A/en active Pending
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101753795B (en) * | 2008-12-16 | 2012-05-16 | 晨星软件研发(深圳)有限公司 | Image processing method for determining motion vector of interpolation image and related device |
| CN101924950B (en) * | 2009-06-17 | 2012-08-29 | 普诚科技股份有限公司 | Data compression method for intelligent accelerating image display and device |
| CN101621693B (en) * | 2009-07-31 | 2011-01-05 | 重庆大学 | Frame frequency lifting method for combining target partition and irregular block compensation |
| CN102044207B (en) * | 2009-10-26 | 2013-02-06 | 群康科技(深圳)有限公司 | Circuit for adjusting setting time and holding time of driving chip |
| CN102111613B (en) * | 2009-12-28 | 2012-11-28 | 中国移动通信集团公司 | Image processing method and device |
| CN102263925A (en) * | 2010-05-31 | 2011-11-30 | 郑朝钟 | Occlusion processing method |
| CN102123283A (en) * | 2011-03-11 | 2011-07-13 | 杭州海康威视软件有限公司 | Interpolated frame acquisition method and device in video frame rate conversion |
| CN102123283B (en) * | 2011-03-11 | 2013-10-30 | 杭州海康威视数字技术股份有限公司 | Interpolated frame acquisition method and device in video frame rate conversion |
| CN102447902A (en) * | 2011-09-30 | 2012-05-09 | 曾志华 | Method for selecting reference field and acquiring time-domain motion vector |
| CN102595089A (en) * | 2011-12-29 | 2012-07-18 | 香港应用科技研究院有限公司 | Frame-rate conversion using mixed bidirectional motion vector for reducing corona influence |
| CN102595089B (en) * | 2011-12-29 | 2014-01-29 | 香港应用科技研究院有限公司 | Frame-rate conversion using mixed bidirectional motion vector for reducing corona influence |
| CN102523419A (en) * | 2011-12-31 | 2012-06-27 | 上海大学 | Digital video signal conversion method based on motion compensation |
| CN105100807A (en) * | 2015-08-28 | 2015-11-25 | 山东大学 | Motion vector post-processing based frame rate up-conversion method |
| CN105100807B (en) * | 2015-08-28 | 2018-11-13 | 山东大学 | A kind of frame per second method for improving based on motion vector post-processing |
| CN110290360B (en) * | 2019-08-01 | 2021-02-23 | 浙江开奇科技有限公司 | Image stitching method and terminal equipment for panoramic video image |
| CN110290360A (en) * | 2019-08-01 | 2019-09-27 | 浙江开奇科技有限公司 | Image splicing method and terminal device for panoramic video image |
| CN110933496A (en) * | 2019-12-10 | 2020-03-27 | Oppo广东移动通信有限公司 | Image data frame insertion processing method and device, electronic equipment and storage medium |
| CN111327926A (en) * | 2020-02-12 | 2020-06-23 | 北京百度网讯科技有限公司 | Video frame insertion method and device, electronic equipment and storage medium |
| CN111327926B (en) * | 2020-02-12 | 2022-06-28 | 北京百度网讯科技有限公司 | Video frame insertion method and device, electronic equipment and storage medium |
| CN111614911A (en) * | 2020-05-08 | 2020-09-01 | Oppo广东移动通信有限公司 | Image generation method and device, electronic equipment and storage medium |
| CN111614911B (en) * | 2020-05-08 | 2022-12-09 | Oppo广东移动通信有限公司 | Image generation method and device, electronic device and storage medium |
| CN111641829A (en) * | 2020-05-16 | 2020-09-08 | Oppo广东移动通信有限公司 | Video processing method, device, system, storage medium and electronic equipment |
| CN111641829B (en) * | 2020-05-16 | 2022-07-22 | Oppo广东移动通信有限公司 | Video processing method, device and system, storage medium and electronic equipment |
| WO2022267957A1 (en) * | 2021-06-21 | 2022-12-29 | 华为技术有限公司 | Video frame interpolation method and apparatus, and device |
| CN114740965A (en) * | 2022-05-05 | 2022-07-12 | Oppo广东移动通信有限公司 | Processing method and device for reducing power consumption of terminal, terminal and readable storage medium |
| CN114740965B (en) * | 2022-05-05 | 2023-12-08 | Oppo广东移动通信有限公司 | Processing method for reducing terminal power consumption, device thereof, terminal and readable storage medium |
| CN114898577A (en) * | 2022-07-13 | 2022-08-12 | 环球数科集团有限公司 | Road intelligent management system and method for peak period access management |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101207707A (en) | System and method for advancing frame frequency based on motion compensation | |
| US9148622B2 (en) | Halo reduction in frame-rate-conversion using hybrid bi-directional motion vectors for occlusion/disocclusion detection | |
| CN105517671B (en) | Video frame interpolation method and system based on optical flow method | |
| US8446526B2 (en) | Image displaying device and method, and image processing device and method | |
| CN103402098B (en) | A kind of video frame interpolation method based on image interpolation | |
| US8941778B2 (en) | Video display apparatus | |
| CN106210767A (en) | A kind of video frame rate upconversion method and system of Intelligent lifting fluidity of motion | |
| CN101663885A (en) | Image processing device and method, and image display device and method | |
| CN101867759B (en) | Self-adaptive motion compensation frame frequency promoting method based on scene detection | |
| CN101442649B (en) | Processing method and apparatus for removing alternate line and FPGA chip | |
| CN102123283B (en) | Interpolated frame acquisition method and device in video frame rate conversion | |
| CN101754047B (en) | Method for detection of film mode or camera mode | |
| CN102577365B (en) | Video display device | |
| CN103369208A (en) | Self-adaptive de-interlacing method and device | |
| CN102881018B (en) | Method for generating depth maps of images | |
| US20020001347A1 (en) | Apparatus and method for converting to progressive scanning format | |
| CN102447870A (en) | Detection method for static objects and motion compensation device | |
| KR20040049214A (en) | Apparatus and Method for searching motion vector with high speed | |
| CN104202555A (en) | Method and device for deinterlacing | |
| CN103313059B (en) | The decision method of occlusion areas during a kind of frame per second promotes | |
| CN101483790A (en) | Movie mode video signal detection method | |
| JP4470280B2 (en) | Image signal processing apparatus and image signal processing method | |
| KR101359351B1 (en) | Fast method for matching stereo images according to operation skip | |
| CN102075669A (en) | Method and system for adaptively recursively denoising digital video signal | |
| KR102279495B1 (en) | Display apparatus and controlling method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20080625 |