CN101778292B - Image processing method and device thereof - Google Patents
Image processing method and device thereof Download PDFInfo
- Publication number
- CN101778292B CN101778292B CN 200910000611 CN200910000611A CN101778292B CN 101778292 B CN101778292 B CN 101778292B CN 200910000611 CN200910000611 CN 200910000611 CN 200910000611 A CN200910000611 A CN 200910000611A CN 101778292 B CN101778292 B CN 101778292B
- Authority
- CN
- China
- Prior art keywords
- vector
- block
- image
- motion
- interpolation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention discloses an image processing method and a device thereof, which can ensure or enhance the image quality. The image processing method is used for deciding a motion vector of an image covering area/image displaying area of an interpolation picture when moving picture interpolation, the interpolation picture consists of a plurality of blocks, and the method comprises the steps of producing a first motion vector and a second motion vector of a block in the interpolation picture, determining the block in one of the image covering area and the image displaying area according to the first motion vector and the second motion vector, computing a reference vector, and determining the motion vector of the block according to the reference vector, wherein the reference vector is obtained after vector operation is carried out to the first motion vector and the second motion vector.
Description
Technical field
The present invention relates to a kind of image processing mechanism, refer to a kind of in order to determine the image area of coverage/image in the interpolation image to appear image treatment method and the relevant apparatus thereof of the image in district especially.
Background technology
During the motion-vector of the interpolation block of present traditional image interpolation mechanism in determining an interpolation image, system directly decides its motion-vector with the operation result of block alignment algorithm (block matching algorithm), to produce the image of this interpolation block.Please refer to Fig. 1, it is the schematic diagram of block alignment algorithm.As shown in Figure 1, image frame F2, F3 are forward and backward two picture in the input image, and picture F
InterIt is the interpolation image that traditional image interpolation mechanism produces, A '～L ' is the background video that represents among image frame F2, the F3, the direction that dotted arrow system expression background video among the figure moves, solid arrow is then represented the direction that a foreground object moves, the background video of F ' in image frame F2～I ' is covered in by this foreground object, then is that the background video of C '～F ' is covered in by this foreground object in next image frame F3; Because the block alignment algorithm can all can find background video A ', B ', J ', K ', L ' in forward and backward two image frame F2, F3, so, can determine the correct target motion-vector of corresponding block, and background video A ', B ', J ', K ', L ' are presented in picture F
InterGo up (as shown in Figure 1); In addition, the block alignment algorithm also can find the image of foreground object in forward and backward two image frame F2, F3, therefore, can determine the correct target motion-vector of corresponding block, and the image of foreground object is presented in picture F
InterOn.
Yet, determining regional R
InterWith R
Inter' in the target motion-vector of interpolation block the time, regional R ideally
InterWith R
Inter' in the target motion-vector of interpolation block should be that the background motion-vector is so that regional R
InterWith R
Inter' in can present background video, for example, regional R
InterShould present background video C ', D ' ideally, and regional R
Inter' should present background video H ', I ' ideally, yet, in fact, because the block alignment algorithm can not find background video C ', D ' (being covered by foreground object) in image frame F3, and in image frame F2, can not find background video H ', I ' (originally covered by foreground object), so the block alignment algorithm also can't determine correct motion-vector, cause interpolation zone R
Inter, R
Inter' the actual image distortion that presents, when being applied to the conversion of picture frame renewal frequency, traditional image interpolation mechanism will significantly reduce the quality of image output.
Summary of the invention
Technical problem to be solved by this invention provides a kind of image treatment method and device thereof, can guarantee or promote the quality of image.
In order to solve above technical problem, the invention provides following technical scheme:
The invention provides a kind of image treatment method.This image treatment method system determines the image area of coverage/image of an interpolation image to appear the motion-vector in district when being used for carrying out the motion picture interpolation, this interpolation image is made up of plurality of blocks, and this method includes: one first motion-vector and one second motion-vector that produce a block in this interpolation image; According to this first motion-vector and this second motion-vector, determine this block to be positioned at one of them that the image area of coverage and this image appear the district, and calculate a reference vector; And according to this reference vector, determine the motion-vector of this block; Wherein, this system of vectors of reference carries out a vector operation gained to this first, second motion-vector.
The invention provides a kind of image treatment method.When this image treatment method system is used for carrying out the motion picture interpolation image area of coverage/image of one interpolation image is appeared the district and carry out the image interpolation, this interpolation image is made up of plurality of blocks, and this method includes: utilize block comparison method, calculate the motion-vector of plurality of blocks in this interpolation image, last interpolation image and back one interpolation image; According to the motion-vector of a block in this interpolation image and the motion-vector of contiguous plurality of blocks, produce one first motion-vector and one second motion-vector of this block in this interpolation image, wherein the motion-vector of this block is first candidate vector; According to this first motion-vector and this second motion-vector, calculate a reference vector of this block; According to this reference vector, be that an initial position points to a comparison block in last interpolation image or back one interpolation image with this block in this interpolation image, and be second candidate vector with the motion-vector of this comparison block in this last interpolation image or this back one interpolation image; According to this first, second candidate vector, carry out the image interpolation of this block.
The invention provides a kind of image processor.This image processor system determines the image area of coverage/image of an interpolation image to appear the motion-vector in district when being used for carrying out the motion picture interpolation, this interpolation image is made up of plurality of blocks, and this image processor includes a storage element and a computing unit, wherein this storage element system is used for storing at least this interpolation image, the original motion-vector of relevant block in last interpolation image and back one interpolation image, and this computing unit is coupled to this storage element, and comprise a motion-vector generation module and a motion-vector decision module, wherein this motion-vector generation module system is for one first motion-vector and one second motion-vector of a block that decides this interpolation image, and this motion-vector decision module system is according to this first motion-vector and this second motion-vector, determine this block to be positioned at one of them that the image area of coverage and this image appear the district, and calculate a reference vector, and determine the motion-vector of this block according to this; In addition, this system of vectors of reference carries out a vector operation gained to this first, second motion-vector.
The invention provides a kind of image processor.When this image processor system is used for carrying out the motion picture interpolation image area of coverage/image of one interpolation image is appeared the district and carry out the image interpolation, this interpolation image is made up of plurality of blocks, and this image processor includes a storage element and a computing unit, wherein this storage element system is used for storing at least this interpolation image, the original motion-vector of relevant block in last interpolation image and back one interpolation image, and this computing unit is coupled to this storage element, and include one first candidate vector generation module, one second candidate vector generation module and a picture interpolation module, wherein this first candidate vector generation module system is used for according to the motion-vector of this interpolation image one block and the motion-vector of contiguous plurality of blocks, produce one first motion-vector and one second motion-vector of this block in this interpolation image, wherein the motion-vector of this block is first candidate vector, and this second candidate vector generation module system is used for according to this first motion-vector and this second motion-vector, calculate a reference vector of this block, and according to this reference vector, with this block in this interpolation image be an initial position point to last interpolation image or the back one interpolation image in one the comparison block, and be second candidate vector with the motion-vector of this comparison block in this last interpolation image or this back one interpolation image, and this picture interpolation module system be used for according to this first, second candidate vector is carried out the image interpolation of this block.
The invention provides a kind of image processor, when being used for carrying out the motion picture interpolation, according to an interpolation image, one last interpolation image and one the back one interpolation image in relevant block original motion-vector, to this interpolation image the image area of coverage/image appear the district and carry out the image interpolation, this interpolation image is made up of plurality of blocks, this device includes: one first candidate vector generation module, be used for according to this interpolation image one block motion-vector and contiguous plurality of blocks motion-vector, produce this block in this interpolation image one first motion-vector and one second motion-vector, wherein this block motion-vector be one first candidate vector; One second candidate vector generation module, be used for according to this first motion-vector and this second motion-vector, calculate this block a reference vector, and according to this reference vector, with this block in this interpolation image be an initial position point to last interpolation image or the back one interpolation image in one the comparison block, and with this comparison block in this last interpolation image or this back one interpolation image motion-vector be one second candidate vector; And a picture interpolation module, be used for according to this first candidate vector and this second candidate vector, carry out this block the image interpolation.
Image treatment method and device thereof that the present invention adopts, belong to an image area of coverage or an image appears the district if a block is actual, then above-mentioned image processor and method can produce correct image with reference to second candidate vector of this block, thereby can promote the quality of image, wherein this second candidate vector system is corresponding to the background motion-vector; And if this block actual be not to belong to the image area of coverage or image appears the district, then this image processor and method can produce correct image with reference to first candidate vector of this block, thereby can not influence the quality of image, wherein this first candidate vector is the motion-vector that the block alignment algorithm calculates at this block.
Description of drawings
Fig. 1 is the operation chart of traditional block alignment algorithm.
Fig. 2 is the assembly schematic diagram of the image processor of first embodiment of the invention.
Fig. 3 is the operational flowchart of image processor shown in Figure 2.
The schematic diagram of the image output that Fig. 4 produces when for computing unit shown in Figure 2 the input image being carried out the conversion of picture frame renewal frequency.
Fig. 5 (a) is the example schematic that vectorial generation module shown in Figure 2 calculates the numerical value that represents the motion-vector degree of variation.
Fig. 5 (b) is the example schematic that includes the first numerical curve CV of a plurality of motion-vector variation values.
Fig. 6 (a) is vectorial decision module shown in Figure 2 produces reference vector in another embodiment running schematic diagram.
Fig. 6 (b) is vectorial decision module shown in Figure 2 produces the reference vector of different interpolation blocks in the embodiment of Fig. 6 (a) example schematic.
Fig. 6 (c) is vectorial decision module shown in Figure 2 produces the reference vector of different interpolation blocks in the embodiment of Fig. 6 (a) another example schematic.
Fig. 7 (a) is the embodiment schematic diagram that vectorial decision module shown in Figure 2 determines the second candidate vector MV ' of the interpolation block in the input image shown in Figure 4.
Fig. 7 (b) is another embodiment schematic diagram that vectorial decision module shown in Figure 2 determines the second candidate vector MV ' of the interpolation block in the input image shown in Figure 4.
Fig. 8 is the assembly schematic diagram of the image processor of second embodiment of the invention.
[primary clustering symbol description]
200,800 image processors
205,805 computing units
210,810 storage elements
2051 vectorial generation modules
2052 vectorial decision module
2053,8053 picture interpolation module
8,051 first candidate vector generation modules
8,052 second candidate vector generation modules
8054 select module
Embodiment
At first, be easy-to-read, below be will do not covered by foreground object in last the image frame but in next image frame by background video zone that foreground object covered, be called the image area of coverage (covered area), and will be covered by foreground object in last the image frame but the background video zone of (not covered by foreground object) in next image frame, occurs, be called image and appear district (uncovered area); For instance, background video C ', D ', E ' shown in Figure 1 is the image area of coverage, and background video G ', H ', I ' are that image appears the district; Note that above definition only in order to the operation of convenient explanation embodiments of the invention, is not restriction of the present invention.
Please arrange in pairs or groups with reference to Fig. 2 and Fig. 3, it illustrates the image processor 200 of first embodiment of the invention and the flow process of associative operation step thereof respectively; For convenience of description, the step that below is collocation Fig. 3 illustrates the operation of the image processor 200 of Fig. 2, be noted that, if can reach identical result substantially, the step order that does not need necessarily to shine in the flow process shown in Figure 3 is carried out, and step shown in Figure 3 not necessarily will be carried out continuously, that is other step also can be inserted wherein.Image processor 200 is to determine the image area of coverage/image of an interpolation image to appear the motion-vector in district when carrying out the motion picture interpolation, this interpolation image is made up of plurality of blocks, as shown in Figure 2, image processor 200 includes a computing unit 205 and a storage element 210, wherein storage element 210 is for storing this interpolation image at least, the image data of the original motion-vector of last interpolation image and the block of back in one interpolation image and last of this interpolation image and back two input picture, and computing unit 205 is to be coupled to storage element 210, and include a vectorial generation module 2051, one vectorial decision module 2052 and a picture interpolation module 2053, wherein vectorial generation module 2051 is with the first motion-vector MV that decides/produce the block in this interpolation image (that is first interpolation image)
LWith the second motion-vector MV
R(step 305), in the present embodiment, vectorial generation module 2051 is that each block in this interpolation image is all produced its two corresponding motion-vector MV on real the work
L, MV
R, each block for example includes 8 * 8 pixel coverage, and so this is not restriction of the present invention, and the enforcement modification of other pixel coverage is spirit according to the invention also; In addition, for this block, vectorial decision module 2052 is according to the first motion-vector MV
LWith the second motion-vector MV
ROne of them that decides that this block is positioned at that the image area of coverage and image appear the district also calculated a reference vector (step 310), this reference vector is stored in the storage element 210 again, and wherein this system of vectors of reference is to the first motion-vector MV
LWith the second motion-vector MV
RCarry out a vector operation gained, on real the work, vectorial decision module 2052 is to utilize the first motion-vector MV
LDeduct the second motion-vector MV
RThe vector result that obtains judges that this block is relevant to the image area of coverage or image appears the district.
Then, vector decision module 2052 can be positioned at the result that the image area of coverage or image appear the district according to this reference vector and this block, in last interpolation image or back one interpolation image, find a comparison block (step 315), and the original motion-vector and the block that obtain this comparison block are compared difference (step 320), and when this block was compared difference less than a critical value, vectorial decision module 2052 can be compared this original motion-vector of block as the motion-vector (step 325) of this block with this; At last, picture interpolation module 2053 produces the image (step 330) of this block again with the motion-vector of this block.
On real the work, vector decision module 2052 is to calculate a motion-vector of this block with first candidate vector as this block according to a block alignment algorithm (block matchingalgorithm), and be a starting point according to this reference vector with the locus of this interpolation block in this interpolation image, point to the image block in last interpolation image or back one interpolation image (that is second interpolation image), and calculate according to this block alignment algorithm that this is last/back one interpolation image in a motion-vector of this image block as second candidate vector of this interpolation block, calculating first, after second candidate vector, vector decision module 2052 can be by first, select a vector in second candidate vector as the motion-vector of this block, the motion-vector that the picture interpolation module then determines according to vectorial decision module 2052 produces the image of this interpolation block.
Above-mentioned first, it is the picture that has a non-image interpolation at least between second interpolation image, that is, first interpolation image can be the picture that last image interpolation of second interpolation image produces, for instance, please refer to Fig. 4, its illustrate is (picture frame or figure in 205 pairs of a plurality of input pictures of computing unit shown in Figure 2, only illustrate Fn-2 at this, Fn-1, Fn, Fn+1) carry out picture frame renewal frequency conversion (frame rate conversion) to produce a plurality of image outputs, for example, picture frame Fn-2, Fn-1, Fn, Fn+1 etc. are the input picture picture frame of 60Hz, and computing unit 205 is these input picture picture frames to be carried out the picture frame renewal frequency change to produce the picture frame Fn-2 with 120Hz, F ', Fn-1; F " Fn; F " ', Fn+1 etc.In addition, the image processor 200 of present embodiment also can be applicable to handle the picture frame renewal frequency conversion of different switching ratio after via suitable adjustment, for example be that to become frequency be the image of 240Hz for the video conversion of 60Hz with frequency, in other words, image processor 200 can be between per two image frames the picture frame renewal frequency conversion of reaching different output frequencies of interpolation one or many s' picture.
In fact, computing unit 205 produces an interpolation image F ' to being less than among picture Fn-2, the Fn-1, in picture Fn-1, Fn, produce an interpolation image F "; and in picture Fn, Fn+1, produce an interpolation image F " ', in an embodiment of the present invention, first interpolation image for example is picture F ", and for interpolation image F " in regional R
1", second interpolation image for example is picture F ' (that is interpolation image F " last interpolation image), and for interpolation image F " in regional R
2", second interpolation image for example is picture F " ' (that is interpolation image F " a back interpolation image), in other words, in the present invention, as this interpolation block and the image area of coverage (regional R for example
1") when relevant, this second interpolation image on time sequencing early than this first interpolation image; And appear district (regional R for example when this interpolation block and image
2") when relevant, this second interpolation image is later than this first interpolation image on time sequencing.Image frame Fn-2, Fn-1, Fn, Fn+1 etc. with Fig. 4 are example, and the moving direction system of foreground object moves horizontally from right to left (with motion-vector V
ForeRepresent it), and background video system moves horizontally from left to right (with motion-vector V
BackRepresent it); Be noted that, be only with the example that moves horizontally as an illustration for convenience of description at this, yet, move for the image of arbitrary specific direction (for example vertical direction or picture to angular direction etc.), all can use the image processor 200 of embodiments of the invention to handle.
Following first subtend volume production give birth to module 2051 at each interpolation image (comprise picture F ', F ", F " ' etc.) and each interpolation block produce two motion-vector MV
L, MV
RRunning illustrate.At first, vector generation module 2051 can calculate the original motion-vector of relevant interpolation block according to the block alignment algorithm, again with a plurality of original motion-vector of the original motion-vector adjacent block a plurality of with it that each was calculated, calculate the confusion degree of motion-vector, to calculate a numerical curve, wherein, with present embodiment, this numerical curve is the formed curve of section along the motion-vector confusion degree of a certain interpolation block horizontal direction, and this confusion degree represents the variation value (motion vector variance) of motion-vector, that is this numerical curve includes a plurality of numerical value of the different motion-vector degrees of variation of representative on the specific direction.Please refer to Fig. 5 (a), it is the schematic diagram that the present invention calculates the numerical value that represents the motion-vector degree of variation; For example, if a plurality of original motion-vector that first interpolation block adjacent block a plurality of with it (shown in Fig. 5 (a), being positioned at 5 * 5 block scope) calculate according to the block alignment algorithm is respectively MV
00With MV
-2-2～MV
22Then can calculate a motion-vector variation value MV_VAR according to these original motion-vectors, its algorithm is to get that the maximum horizontal component deducts the wherein absolute value of minimum level component gained in these original motion-vectors, add that maximum vertical component deducts the wherein absolute value of minimum vertical component gained in these original motion-vectors, MV_VAR can utilize following equation to represent it:
MV_VAR=|MAX (MV
x)-MIN (MV
x) |+| MAX (MV
y)-MIN (MV
y) | equation (1)
Wherein MVx and MVy represent horizontal component (x axle component) and vertical component (y axle component) respectively.Be noted that 5 * 5 block scope is not to be restriction of the present invention, it also can utilize the block scope of N * N or N * M to do it in fact, and wherein Parameter N and M all are that positive integer and N are not equal to M; In addition, the mode of calculating motion-vector variation value MV_VAR also can be used equation (2) instead or equation (3) is realized:
MV_VAR＝|MAX(MV
x)-MIN(MV
x)|+|MAX(MV
y)-MIN(MV
y)|+SAD
Equation (2)
MV_VAR＝α×{MAX(MV
x)-MIN(MV
x)|+|MAX(MV
y)-MIN(MV
y)|}+β×SAD
Equation (3)
Wherein numerical value SAD is the block comparison difference that the first interpolation block is calculated according to block comparison method, and parameter alpha, β are weighting parameters; All can variation in order to arbitrary enforcement of calculating the numerical value that represents the motion-vector degree of variation all belongs to category of the present invention.From the above mentioned, according to equation (1), equation (2) or equation (3) one of them, vectorial generation module 2051 calculates at different interpolation blocks one by one, so can draw the first numerical curve CV, shown in Fig. 5 (b).
Fig. 5 (b) illustrate is for being the first numerical curve CV that the present invention includes a plurality of motion-vector variation values.MB
00Be the first interpolation block, and vectorial generation module 2051 is to determine interpolation block MB in Fig. 5 (b)
00Two motion-vector MV
L, MV
R, at first, with the direction (for example horizontal direction) that background or prospect image move, vectorial generation module 2051 is spatially prolonging the first interpolation block MB
00Both sides a plurality of (for example both sides each six, MB
10～MB
60With MB
-10～MB
-60) the corresponding motion-vector variation value of block, in these corresponding motion-vector variation value scopes, take out a maximum (VAR shown in Fig. 5 (b) for example
Max), and in these corresponding motion-vector variation value scopes, find out this maximum VAR
MaxThe block that the minimum of and arranged on left and right sides corresponds to for example, can find block MB
-40With MB
50, and these left and right two block MB
-40With MB
50Utilize motion-vector that the block alignment algorithm calculates namely as interpolation block MB respectively
00Two motion-vector MV
L, MV
R, in other words, interpolation block MB
00The first motion-vector MV
LSystem corresponds to the maximum VAR that is positioned at the first numerical curve CV
MaxThe minimum VAR in left side
Min, and its second motion-vector MV
RSystem corresponds to the maximum VAR that is positioned at the first numerical curve CV
MaxThe minimum VAR on right side
Min'; First, second motion-vector MV
L, MV
ROne of them be to correspond to background motion-vector (background motion vector), its another then corresponding to prospect motion-vector (foreground motion vector), this is because the motion-vector variation value that belongs to around the interpolation block that the image area of coverage or image appear the district will be quite big, and the minimum motion-vector variation of and arranged on left and right sides is worth corresponding image block, it can be corresponding to a prospect motion-vector or a background motion-vector, looks it and is positioned at the image area of coverage or image appears Qu Erding.Therefore, if interpolation block MB
00System is positioned at one of them that the image area of coverage and image appear the district, then its first, second motion-vector MV
L, MV
ROne of them corresponding to the background motion-vector, and its another corresponding to the prospect motion-vector.By above-mentioned operation, vectorial generation module 2051 can calculate the corresponding two motion-vector MV to each interpolation block among the interpolation image F '
LWith MV
R
Producing motion-vector MV
L, MV
RAfterwards, vectorial decision module 2052 can be according to motion-vector MV
L, MV
RJudge in this interpolation image whether an interpolation block belongs in the image area of coverage (or image appears the district) or its adjacent domain (that is, judge that whether this interpolation block is to be relevant to the image area of coverage or image appears the district), then according to the motion-vector MV of each interpolation block
L, MV
RCalculate the reference vector of each interpolation block with above-mentioned judged result.For convenience of description, below be (comprising picture frame F1, F2, F3) be the mode that example illustrates present embodiment generation reference vector with another input image; Please refer to Fig. 6 (a), its illustrate produces the running schematic diagram of reference vector in another embodiment into vectorial decision module 2052 shown in Figure 2.Shown in Fig. 6 (a), with horizontal direction, the foreground object image is static (with motion-vector V
Fore' expression), and background video system moves horizontally from right to left (with motion-vector V
Back' expression) because present embodiment mainly is that to move horizontally image be that example illustrates, therefore with motion-vector direction from left to right for just, otherwise its relative direction (from right to left) is then for bearing; Yet, this non-restriction of the present invention, all on the two dimensional surface space with a certain specific direction for just, change spirit all according to the invention with its relative direction for negative enforcement again.
On real the work, the mode that produces the reference vector of an interpolation block is with motion-vector MV
L, MV
RDecide the vector magnitude of reference vector and utilize above-mentioned judged result to decide the direction of reference vector.For the vector magnitude of reference vector, the block system on the position P1 shown in Fig. 6 (a) is positioned at foreground object image and image area of coverage R
1The edge, and its first, second candidate vector MV
L, MV
RBe respectively prospect motion-vector and background motion-vector, shown in Fig. 6 (a), first, second candidate vector MV
L, MV
RCan form leg-of-mutton both sides, its 3rd limit falls within on the picture F2, according to the similar triangles basic principle, and image area of coverage R
1The image capturing range size system that comprises is regional R among the picture F2 just
1' half of the image capturing range size that comprises, and vectorial decision module 2052 is got motion-vector MV
LHalf and motion-vector MV
RHalf carry out the vectorial addition computing, can calculate a vectorial V
Ref, vectorial V
RefVector magnitude namely be the vector magnitude of reference vector; And for the direction of reference vector, vectorial decision module 2052 is with motion-vector MV
LDeduct motion-vector MV
RAfter the vector result that obtains judge that whether this interpolation block is to be relevant to the image area of coverage or image appears the district, carry out oppositely to obtain this reference vector to determine the vector that whether this vectorial addition result is produced, be with vectorial V in this example
RefOppositely obtain reference vector Vref ', as seen from the figure, reference vector Vref ' is a starting point with position P1, can point to interpolation image F
aPosition P2, and position P2 is image area of coverage R
2And the marginal position between the background video; Below in Fig. 6 (b) and Fig. 6 (c) will be at Fig. 6 (a) different interpolation blocks describe it.
At first, with the interpolation image F between image frame F2, the F3
bBe example, in fact belonging to image area of coverage R
1The interpolation block (for example near the interpolation block MB of foreground object image
1), its motion-vector MV
L, MV
RBe respectively prospect motion-vector and background motion-vector, according to above-mentioned calculating, vectorial decision module 2052 is got motion-vector MV
LHalf and motion-vector MV
RHalf carry out the vectorial addition computing, can calculate a vectorial V
1, and vectorial decision module 2052 can be with motion-vector MV
LDeduct motion-vector MV
RObtain a vector result, because this vector result is just (to represent direction from left to right), so vectorial decision module 2052 is this interpolation block MB as can be known
1Be relevant with the image area of coverage, and determine interpolation block MB according to this vector result
1The follow-up reference vector V that is calculated
1Should point to last interpolation image F
a, that is, vectorial decision module 2052 according to this vector result with vectorial V
1Carry out oppositely producing interpolation block MB as shown in Fig. 6 (b)
1Reference vector V
1, reference vector V wherein
1With interpolation block MB
1For starting point can point to last interpolation image F
aMiddle block MB
1' the position, this is because the reference vector Vref ' shown in Fig. 6 (a) means position P2, then the reference vector V shown in Fig. 6 (b) relatively
1Mean block MB
1', and block MB
1' in fact be not belong to interpolation image F
aIn image area of coverage R
2, and block MB
1' motion-vector of calculating according to the block alignment algorithm is the background motion-vector; In like manner, in fact belonging to image area of coverage R
1Interpolation block MB
2(it is near background video but in fact still belong to image area of coverage R
2), vectorial decision module 2052 also can be calculated its reference vector V according to aforesaid way
2With interpolation block MB
2For starting point means interpolation image F
aIn block MB
2' the position, block MB
2' in fact be not belong to interpolation image F
aIn image area of coverage R
2, and block MB
1' motion-vector of calculating according to the block alignment algorithm is the background motion-vector.
In addition, in fact close image footprint edges R
1But be the interpolation block MB that belongs to the prospect image
3, its motion-vector MV
L, MV
RBe respectively prospect motion-vector and background motion-vector, so it should be noted that and comply with the reference vector V that calculates
3With interpolation block MB
3For starting point will point to interpolation image F
aIn in fact belong to image area of coverage R
2Block MB
3' the position.In addition, in fact close image area of coverage R
1Edge but be the interpolation block MB that belongs to background video
4, its motion-vector MV
L, MV
RBe respectively prospect motion-vector and background motion-vector, so it should be noted that and comply with the reference vector V that calculates
4With interpolation block MB
4For starting point will point to interpolation image F
aIn in fact belong to the block MB of background video
4' the position.Moreover, in fact not belonging to image area of coverage R
1And do not fall within the interpolation block MB of its adjacent domain
5, interpolation block MB
5Two motion-vector MV
L, MV
RBe respectively the prospect motion-vector, vectorial decision module 2052 is with its motion-vector MV
LDeduct motion-vector MV
RAfter the vector result that obtains almost nil (expression is not to belong to the image area of coverage and contiguous zone thereof), when vector result is almost nil, reference vector V
5(reference vector V
5Vector magnitude be result after two prospects vector each half carries out add operation, therefore still represent the vector magnitude of prospect vector) with interpolation block MB
5For starting point points to last interpolation image F
aAn interpolation block MB
5' the locus or the locus (not being shown among Fig. 6 (b)) of an interpolation block of a back interpolation image can't have influence on subsequent operation, so vectorial decision module 2052 still can judge whether according to the plus or minus of this vector result vectorial addition results is carried out oppositely to produce reference vector V
5, in present embodiment, vectorial decision module 2052 is that vectorial addition results is carried out oppositely to produce reference vector V
5In like manner, in fact not belonging to image area of coverage R
1And do not fall within the interpolation block MB of its adjacent domain
6, interpolation block MB
6Two motion-vector MV
L, MV
RBe respectively the background motion-vector, vectorial decision module 2052 is with its motion-vector MV
LDeduct motion-vector MV
RAfter the vector result that obtains also almost nil, when vector result is almost nil, reference vector V
6With interpolation block MB
6For starting point points to last interpolation image F
aIn a block MB
6' the position or the position (not being shown among Fig. 6 (b)) of a block of a back interpolation image can't have influence on subsequent operation, so vectorial decision module 2052 still can judge whether according to the plus or minus of this vector result vectorial addition results is carried out oppositely to produce reference vector V
6, in present embodiment, vectorial decision module 2052 is that vectorial addition results is carried out oppositely to produce reference vector V
6, reference vector V wherein
6Vector magnitude be result after two backgrounds vector each half carries out add operation, therefore still represent the vector magnitude of background vector.
Then, please refer to Fig. 6 (c), with interpolation image F
aBe example, appear district R in fact belonging to image
3The interpolation block (for example near the interpolation block MB of foreground object image
7), its motion-vector MV
L, MV
RBe respectively background motion-vector and prospect motion-vector, vectorial decision module 2052 is got motion-vector MV
LHalf and motion-vector MV
RHalf carry out the vectorial addition computing, can calculate a vectorial V
7(shown in Fig. 6 (c)), and vectorial decision module 2052 can be with motion-vector MV
LDeduct motion-vector MV
RObtain a vector result, because this vector result is for negative (expression direction from right to left), so vectorial decision module 2052 is this interpolation block MB as can be known
7System appears district R with image
3Relevant, and according to this vector result decision interpolation block MB
7The follow-up reference vector of being calculated should point to a back picture F
b, that is, vectorial decision module 2052 according to this vector result with vectorial V
7As the reference vector and not in addition to vectorial V
7Carry out oppositely, therefore, vectorial decision module 2052 is utilized the above-mentioned reference vector V that calculates
7With interpolation block MB
7The position be starting point, can point to interpolation image F
bIn in fact do not belong to image and appear the block MB that distinguishes R4
7' the position, block MB wherein
7' motion-vector of calculating according to the block alignment algorithm is the background motion-vector; In like manner, appear district R in fact belonging to image
3Interpolation block MB
8(it is near background video but still belong to image and appear district R
3), vectorial decision module 2052 also can be calculated its reference vector V according to aforesaid way
8With interpolation block MB
8' the position be starting point, mean interpolation image F
bIn block MB
8' the position, shown in Fig. 6 (c).
In addition, in fact appearing area edge but be the interpolation block MB that belongs to the prospect image near image
9, its motion-vector MV
L, MV
RBe respectively background motion-vector and prospect motion-vector, so it should be noted that and comply with the reference vector V that calculates
9With interpolation block MB
9The position be starting point, will point to interpolation image F
bIn in fact belong to image and appear the block MB that distinguishes R4
9' the position.In addition, in fact appearing area edge but be the interpolation block MB that belongs to background video near image
10, its motion-vector MV
L, MV
RBe respectively background motion-vector and prospect motion-vector, so it should be noted that and comply with the reference vector V that calculates
10With interpolation block MB
10The position be starting point, will point to interpolation image F
bIn in fact belong to the block MB of background video
10' the position.Moreover, do not appear district R in fact not belonging to image
3And do not fall within the interpolation block MB of its adjacent domain
11, interpolation block MB
11Two motion-vector MV
L, MV
RBe respectively the background motion-vector, vectorial decision module 2052 is with its motion-vector MV
LDeduct motion-vector MV
RAfter the vector result that obtains almost nil (expression is not to belong to image to appear district R
3And contiguous zone), when vector result is almost nil, reference vector V
11With interpolation block MB
11The position be position (not being shown among Fig. 6 (c)) or the back interpolation image F that starting point points to a block of last interpolation image
bA block MB
11' the position can't have influence on subsequent operation, so, vector decision module 2052 still can judge whether according to the plus or minus of this vector result vectorial addition results is oppositely produced reference vector, in present embodiment, vectorial decision module 2052 is that vectorial addition results is carried out oppositely to produce reference vector V
11, reference vector V wherein
11Vector magnitude be result after two backgrounds vector each half carries out add operation, therefore still represent the vector magnitude of background vector.
Please refer to Fig. 7 (a), its illustrate determines the embodiment of the second candidate vector MV ' of an interpolation block in the input image shown in Figure 4 into vectorial decision module 2052; Be noted that the first candidate vector MV of each interpolation block is the motion-vector that vectorial decision module 2052 is calculated according to the block alignment algorithm in the present invention.Shown in Fig. 7 (a), foreground object image system moves horizontally from right to left (with motion-vector V
ForeRepresent), and background video system moves horizontally from left to right (with motion-vector V
BackExpression), the regional R among the interpolation image F '
A' actual be the image area of coverage, and interpolation image F " in regional R
A" reality also is the image area of coverage; With regional R
A" interior interpolation block MB
ABe example, vectorial decision module 2052 is according to aforesaid way generation reference vector VA and with interpolation block MB
AThe position be that starting point is pointed to the block MB among the interpolation image F '
A' the position, the block MB that calculates according to the block alignment algorithm
A' motion-vector namely be interpolation block MB
AThe second candidate vector MV ', its first candidate vector MV then is its interpolation block MB
AThe motion-vector that itself calculates according to the block alignment algorithm, shown in Fig. 7 (a), the block MB that reference vector VA points to
A' in fact be not belong to image area of coverage R
A', the block alignment algorithm can find same or analogous image (image block MB in image frame Fn-2 and Fn-1
N-2With MB
N-1)), therefore, compare difference (that is image block MB according to the block that the second candidate vector MV ' calculates
N-2With MB
N-1) difference), because its value system is quite little, vectorial decision module 2052 can be judged interpolation block MB
AThe second candidate vector MV ' be corresponding to the background motion-vector, on real the work, vectorial decision module 2052 relatively this block comparison difference of calculating and a certain threshold TH reaches and judges that the second candidate vector MV ' is whether corresponding to the operation of background motion-vector.
In addition, with in fact close image area of coverage R among Fig. 7 (a)
A" but be the interpolation block MB of prospect image
BBe example, its reference vector VB that calculates is shown in Fig. 7 (a), with interpolation block MB
BThe position be starting point, mean in interpolation image F ' to belong to image area of coverage R
A' block MB
B' the position, the block MB that calculate according to the block alignment algorithm this moment
B' motion-vector namely be interpolation block MB
BThe second candidate vector MV ', its first candidate vector MV then is interpolation block MB
BThe motion-vector that itself calculates according to the block alignment algorithm, in this embodiment, the block MB that reference vector VB points to
B' in fact be to belong to image area of coverage R
A', because block MB among the image frame Fn-2
N-2' image in picture Fn-1, covered by the foreground object image, so the block alignment algorithm can't find same or analogous image in image frame Fn-2 and Fn-1, even if also can not find similar image (block MB according to the background motion-vector
N-2' image be background video and block MB
N-2' the prospect image difference quite big), therefore, the block comparison difference of calculating according to the second candidate vector MV ' can be quite big, vectorial decision module 2052 can be judged interpolation block MB
BThe second candidate vector MV ' be not the background motion-vector, on real the work, vector decision module 2052 is this block comparison difference and certain threshold TH that calculates relatively, learn that this block comparison difference is not less than certain threshold TH, thereby judge that the second candidate vector MV ' is not is the background motion-vector.
In the embodiment of Fig. 7 (a), with interpolation block MB
A, the image block MB that calculates according to this second candidate vector MV ' among the interpolation image F '
A' block comparison difference system less than certain threshold TH, vectorial decision module 2052 directly uses the second candidate vector MV ' as interpolation block MB
AThe target motion-vector, picture interpolation module 2053 is the above-mentioned judged result of foundation and produce interpolation block MB with reference to this target motion-vector then
AImage; Yet, with interpolation block MB
B, the image block MB that calculates according to its second candidate vector MV ' among the interpolation image F '
B' block comparison difference system be not less than certain threshold TH, vectorial decision module 2052 is to use its first candidate vector MV as interpolation block MB
BThe target motion-vector, picture interpolation module 2053 then produces interpolation block MB with reference to this target motion-vector
BImage.Its reason as previously mentioned, when the block that calculates according to the second candidate vector MV ' is compared difference less than certain threshold TH, vector decision module 2052 can judge that just the second candidate vector MV ' is the background motion-vector, and adopt the second candidate vector MV ' as the target motion-vector of interpolation block, and picture interpolation module 2053 then copies the image that a specific image is used as this interpolation block with this target motion-vector to last input picture image or a back input picture image, for instance, vectorial decision module 2052 is according to interpolation block MB
A(or MB
B) motion-vector MV
L, MV
RPoor, can judge this interpolation block MB
A(or MB
B) to appear the district relevant with the image area of coverage or image, with interpolation block MB
AWith MB
B, vectorial decision module 2052 can judge that two interpolation blocks are all relevant with the image area of coverage, yet different is, the image block MB that calculates according to the corresponding second candidate vector MV '
A' block comparison difference system less than certain threshold TH, and the image block MB that calculates according to the corresponding second candidate vector MV '
B' block comparison difference system be not less than certain threshold TH, so picture interpolation module 2053 can adopt interpolation block MB
AThe second candidate vector MV ' (it is the background motion-vector) copy corresponding image as interpolation block MB to last input picture image Fn-1
AImage, and adopt interpolation block MB
BThe first candidate vector MV (it is the prospect motion-vector) copy corresponding image as interpolation block MB to last input picture image Fn-1
BImage, in other words, interpolation block MB
BLast system suitably presents the image of foreground object, and interpolation block MB
ALast system suitably presents background video, and the quality of image can obtain to promote.
In addition, in a second embodiment, also can be by the image that synthesizes the different images block according to first, second candidate vector MV, MV ' to produce the image of this interpolation block; Please refer to Fig. 8, it illustrates the image processor 800 of second embodiment of the invention.Image processor 800 is to determine the image area of coverage/image of an interpolation image to appear the motion-vector in district when carrying out the motion picture interpolation, it includes a computing unit 805 and a storage element 810, wherein the function of storage element 810 is the storage element 210 that is similar to Fig. 2 with operation, so do not give unnecessary details in addition at this, computing unit 805 then includes one first candidate vector generation module 8051, one second candidate vector generation module 8052 and a picture interpolation module 8053, wherein the first candidate vector generation module 8051 is according to the motion-vector of a block in this interpolation image and the motion-vector of contiguous plurality of blocks, produces one first motion-vector MV of this block in this interpolation image
LWith one second motion-vector MV
R, and the motion-vector of this block is the first candidate vector MV, 8052 of the second candidate vector generation modules are according to this first motion-vector MV
LWith this second motion-vector MV
RCalculate a reference vector of this block, and according to this reference vector, with this block in this interpolation image be an initial position point to last interpolation image or the back one interpolation image in one the comparison block, and with this last interpolation image or should back one interpolation image in should the comparison block motion-vector as the second candidate vector MV ', and picture interpolation module 8053 is to carry out the image interpolation of this block according to this first, second candidate vector MV, the MV ' that determine.
For instance, for interpolation block MB
A, picture interpolation module 8053 is with reference to interpolation image F according to its first candidate vector MV " last input picture Fn-1 in corresponding image (or with reference to interpolation image F " last input picture Fn-1 and the corresponding image of a back input picture Fn) to produce one first image F
1st, and according to its second candidate vector MV ' with reference to another the corresponding image among the last input picture Fn-1 to produce one second image F
2nd, particularly, picture interpolation module 8053 is to utilize first, second candidate vector MV, MV ' to copy corresponding image among the image frame Fn-1 respectively as first, second image F
1st, F
2nd, this moment is because known interpolation block MB
AThe second candidate vector MV ' be the background motion-vector, so picture interpolation module 8053 owners will be with the second image F
2ndProduce interpolation block MB
AImage, will use above-mentioned weighted value (weighting value) to reach on real the work, for example, for interpolation block MB
A, weighted value W
ASize system according to the block MB that calculates with reference to the second candidate vector MV ' among the interpolation image F '
A' block comparison difference decide, in addition, and also the relation that this weighted value and this block are compared between the difference can be designed to a linear relationship, when block comparison difference is big, design this weighted value and become bigger, and when block comparison difference hour, design this weighted value and become less.8053 of picture interpolation module are according to weighted value W
ATo the first image F
1stWith the second image F
2ndCarry out the weighted average of pixel, this average result will be mainly the second image F
2ndPixel value (because this block comparison difference is quite little, weighted value W
ALess relatively), and this average result system is as interpolation block MB
AImage; Interpolation block MB
AImage can utilize equation (4) to represent it:
MB
A=W
A* F
1st+ (1-W
A) * F
2ndEquation (4)
In addition, in another embodiment, for interpolation block MB
B, picture interpolation module 8053 also utilizes its corresponding first, second candidate vector MV, MV ' to copy corresponding image among the image frame Fn-1 respectively as first, second image F
1st, F
2nd, this moment is because known interpolation block MB
BThe second candidate vector MV ' be not to be the background motion-vector, so picture interpolation module 8053 owners will be with the first image F
1stProduce interpolation block MB
BImage, this moment according to the block MB that calculates with reference to its second candidate vector MV ' among the interpolation image F '
B' block comparison difference quite big, its corresponding weighted value W
BIt is bigger to become, and picture interpolation module 8053 is according to weighted value W
BTo the first image F
1stWith the second image F
2ndCarry out the weighted average of pixel, this average result will be mainly the first image F
1stPixel value (because this block comparison difference is quite big, weighted value W
BRelatively large), and this average result system is as interpolation block MB
BImage; Interpolation block MB
BImage can utilize equation (5) to represent it:
MB
B=W
B* F
1st+ (1-W
B) * F
2ndEquation (5)
In addition, in another embodiment, the mode that the image of synthetic different images block produces this interpolation block image also can be arranged in pairs or groups and be utilized this target motion-vector to produce the mode of this interpolation block image; For example, can adopt two critical value TH
1, TH
2Reach above-mentioned purpose, wherein critical value TH
2Greater than critical value TH
1Block in the second interpolation image F ' that the second candidate vector MV ' of an interpolation block points to, its block that calculates according to the block alignment algorithm is compared difference less than critical value TH
1The time, picture interpolation module 8053 is with the target motion-vector of the second candidate vector MV ' as this interpolation block, wherein because the second image F
2ndAlmost be equal to the corresponding image that utilizes this target motion-vector to copy to the image frame Fn-1, so can treat as this moment this weighted value be designed to zero; And work as this block comparison difference more than or equal to critical value TH
2The time, picture interpolation module 8053 is with the target motion-vector of the first candidate vector MV as this interpolation block, wherein because the first image F
1stAlmost be equal to the corresponding image that utilizes this target motion-vector to copy to the image frame Fn-1, thereby can treat as this moment this weighted value is designed to 1; Fall into critical value TH and compare difference when this block
1, TH
2Between the time, 8053 of picture interpolation module are by synthetic first, second image F
1st, F
2ndMode produce the image of this interpolation block, too tediously long for exempting from length, do not give unnecessary details its calculating process in addition at this.All above-mentioned arbitrary enforcement that is used for producing the image of interpolation block changes, and all belongs to category of the present invention.
Please also collocation is with reference to Fig. 7 (b) referring again to Fig. 2, and its illustrate is another embodiment that the vectorial decision module 2052 of Fig. 2 determines the second candidate vector MV ' of an interpolation block in the input image shown in Figure 4; The first candidate vector MV of this interpolation block is the motion-vector that vectorial decision module 2052 is directly calculated according to the block alignment algorithm in the present invention.Shown in Fig. 7 (b), foreground object image system moves horizontally from right to left (with motion-vector V
ForeRepresent), and background video system moves horizontally from left to right (with motion-vector V
BackThe expression), interpolation image F " in regional R
B" actual be that image appears the district, and interpolation image F " ' in regional R
B" ' reality also appears the district for image; With regional R
B" interior interpolation block MB
CBe example, vectorial decision module 2052 is according to aforementioned manner generation reference vector VC and with interpolation block MB
BThe position be that starting point is pointed to interpolation image F " ' in block MB
C' the position, the block MB that calculates according to the block alignment algorithm
C' motion-vector namely be interpolation block MB
CThe second candidate vector MV ', its first candidate vector MV then is interpolation block MB
CThe motion-vector that itself calculates according to the block alignment algorithm, wherein, because interpolation block MB
CBe positioned at image and appear district R
B" in, the block MB that its reference vector VC is pointed
C' the actual system in position belong to background video and be not positioned at image appear the district R
B" ' in, directly use the block alignment algorithm to come calculation block MB
C' motion-vector the time can in image frame Fn and Fn+1, find same or analogous image (image block MBn and MBn+1), therefore, according to the block comparison difference (that is difference of image block MBn and MBn+1) that the second candidate vector MV ' calculates, vectorial decision module 2052 can be judged interpolation block MB
CThe second candidate vector MV ' (that is the block MB that directly uses the block alignment algorithm to calculate
C' motion-vector) be the background motion-vector, on real the work, vectorial decision module 2052 relatively this block comparison difference of calculating and certain threshold TH reaches and judges whether the second candidate vector MV ' is the operation of background motion-vector.
In addition, in fact to be positioned at the interpolation block MB of prospect imagery zone among Fig. 7 (b)
DBe example, the reference vector V that it is calculated
DShown in Fig. 7 (b), with interpolation block MB
DThe position be starting point, mean to interpolation image F " ' in belong to image appear the district R
B" ' an interior block MB
D' the position, the block MB that calculate according to the block alignment algorithm this moment
D' motion-vector system as interpolation block MB
DThe second candidate vector MV ', its first candidate vector MV then is interpolation block MB
DThe motion-vector that itself calculates according to the block alignment algorithm, in this embodiment, interpolation block MB
DBe not positioned at image and appear district R
B" in, but its reference vector V
DBlock MB pointed
D' position reality but be positioned at image appear the district R
B" ' in; the block alignment algorithm can't find same or analogous image in image frame Fn and Fn+1; even if also can not find similar image (image of block MBn ' is that the background video difference of prospect image and block MBn+1 ' is quite big) according to a background motion-vector; so; the block comparison difference of calculating according to the second candidate vector MV ' is quite big, and vectorial decision module 2052 can be judged interpolation block MB
DThe second candidate vector MV ' be not to be the background motion-vector, on real the work, vector decision module 2052 is this block comparison difference and certain threshold TH that calculates relatively, learn that this block comparison difference is not less than certain threshold TH, thereby judge that the second candidate vector MV ' is not is the background motion-vector.
Then, picture interpolation module 2053 can produce interpolation block (block MB for example
COr block MB
D) the target motion-vector obtain the image of this interpolation block, or produce the image of this interpolation block by synthetic different images; This class of operation is similar to the image of the interpolation block shown in aforementioned generation Fig. 7 (a), for the purpose of simplifying the description, does not repeat them here.
Combine it, belong to an image area of coverage or an image appears the district if an interpolation block is actual, then image processor 200 of the present invention, 800 can produce correct image with reference to second candidate vector of this interpolation block, thereby can promote the quality of image, wherein this second candidate vector system is corresponding to the background motion-vector; And if this interpolation block actual be not to belong to the image area of coverage or image appears the district, then image processor 200 of the present invention, 800 can produce correct image with reference to first candidate vector of this interpolation block, thereby can not influence the quality of image, wherein this first candidate vector is the motion-vector that the block alignment algorithm calculates at this interpolation block.
The above only is preferred embodiment of the present invention, and all equalizations of doing according to the present patent application claim change and modify, and all should belong to covering scope of the present invention.
Claims (8)
1. an image treatment method when being used for carrying out the motion picture interpolation, determines the image area of coverage/image of an interpolation image to appear the motion-vector in district, and this interpolation image is made up of plurality of blocks, it is characterized in that, this method includes:
A plurality of original motion-vector according to the original motion-vector adjacent block a plurality of with it of a block produces a numerical curve, wherein this numerical curve by to should block and a plurality of motion-vector variation values of a plurality of adjacent block formed;
Decide first motion-vector and second motion-vector of this block according to this numerical curve, wherein this first motion-vector system corresponds to the original motion-vector of a block of the left side paddy portion of the peak portion that is positioned at this numerical curve, and this second motion-vector system corresponds to the original motion-vector of a block of the right side paddy portion of this peak portion that is positioned at this numerical curve;
Utilize the vectorial difference of first motion-vector and second motion-vector, judge this block be relevant to the image area of coverage or image appear the district one of them, and calculate a reference vector; Be positioned at the result that the image area of coverage or image appear the district according to this reference vector and this block, find a comparison block to be positioned at last interpolation image or back one interpolation image, and the original motion-vector and the block that obtain this comparison block are compared difference; And
When this block comparison difference less than a critical value, the motion-vector that is this block with this original motion-vector of this comparison block then;
Wherein, by half of first motion-vector and half of second motion-vector are carried out the vectorial addition computing to obtain the size of described reference vector; Deduct vector result that second motion-vector obtains according to first motion-vector and judge that whether this block is to be relevant to the image area of coverage or image appears the district, carries out oppositely to obtain this reference vector to determine the vector that whether this vectorial addition computing is produced.
2. image treatment method when being used for carrying out the motion picture interpolation, appearing the district to the image area of coverage/image of an interpolation image and carries out the image interpolation, and this interpolation image is made up of plurality of blocks, it is characterized in that, this method includes:
Utilize block comparison method, calculate the original motion-vector of plurality of blocks in this interpolation image, a last interpolation image and a back one interpolation image;
A plurality of original motion-vector according to the original motion-vector adjacent block a plurality of with it of a block produces a numerical curve, wherein this numerical curve by to should block and a plurality of motion-vector variation values of a plurality of adjacent block formed;
Decide first motion-vector and second motion-vector of this block according to this numerical curve, wherein the motion-vector of this block is one first candidate vector, according to this first motion-vector and this second motion-vector determines this block to be positioned at the image area of coverage and this image appears one of district, wherein this first motion-vector system corresponds to the original motion-vector of a block of the left side paddy portion of the peak portion that is positioned at this numerical curve, and this second motion-vector system corresponds to the original motion-vector of a block of the right side paddy portion of this peak portion that is positioned at this numerical curve;
This first motion-vector and second motion-vector are carried out a vector operation to obtain a reference vector of this block, wherein by half of first motion-vector and half of second motion-vector are carried out the vectorial addition computing to obtain the size of described reference vector; Deduct vector result that second motion-vector obtains according to first motion-vector and judge that whether this block is to be relevant to the image area of coverage or image appears the district, carries out oppositely to obtain this reference vector to determine the vector that whether this vectorial addition computing is produced;
According to this reference vector, be that an initial position points to a comparison block in last interpolation image or back one interpolation image with this block in this interpolation image, and be one second candidate vector with the motion-vector of this comparison block in this last interpolation image or this back one interpolation image; And
Utilize first motion-vector to deduct the vector result that second motion-vector obtains, judge that this block is relevant to the image area of coverage or image appears the district; When this block system is positioned at this image area of coverage, according to this reference vector, be this comparison block that an initial position points to last interpolation image with this block in this interpolation image, a picture interpolation module utilizes last input picture of this interpolation image to carry out the image interpolation; Appearing when district when this block is positioned at image, according to this reference vector, is this comparison block that an initial position points to back one interpolation image with this block in this interpolation image; When this block system was positioned at this image and appears the district, this picture interpolation module utilized a back input picture of this interpolation image to carry out interpolation.
3. image treatment method as claimed in claim 2 is characterized in that, according to this first, second candidate vector, carries out the image interpolation of this block:
When a block that calculates according to this second candidate vector when this comparison block in this last interpolation image or back one interpolation image is compared difference less than a certain threshold, use this second candidate vector as the motion-vector of this block; And
When a block that calculates according to this second candidate vector when this comparison block in this last interpolation image or back one interpolation image is compared difference and is not less than a certain threshold, use this first candidate vector as the motion-vector of this block.
4. image treatment method as claimed in claim 2 is characterized in that, according to this first, second candidate vector, carries out the image interpolation of this block:
Block comparison difference according to this comparison block in last interpolation image or back one interpolation image calculates determines a weighted value;
According to this first candidate vector, second candidate vector and this weighted value, carry out the image interpolation of this block.
5. image processor when being used for carrying out the motion picture interpolation, determines the image area of coverage/image of an interpolation image to appear the motion-vector in the district, and this interpolation image is made up of plurality of blocks, it is characterized in that, this device includes:
One storage element is for the original motion-vector of the relevant block that stores this interpolation image, a last interpolation image and a back one interpolation image at least; And
One computing unit is coupled to this storage element, includes:
One vectorial generation module, the a plurality of original motion-vector of the one original motion-vector adjacent block a plurality of with it of foundation one block, produce a numerical curve, wherein this numerical curve by to should block and a plurality of motion-vector variation values of a plurality of adjacent block formed;
Decide first motion-vector and second motion-vector of this block according to this numerical curve, wherein this first motion-vector system corresponds to the original motion-vector of a block of the left side paddy portion of the peak portion that is positioned at this numerical curve, and this second motion-vector system corresponds to the original motion-vector of a block of the right side paddy portion of this peak portion that is positioned at this numerical curve;
One vectorial decision module, utilize first motion-vector to deduct the vector result that second motion-vector obtains, judge that this block is relevant to the image area of coverage or image appears the district, and determine this block to be positioned at one of them that the image area of coverage and this image appear the district, be positioned at the result that the image area of coverage or image appear the district according to a reference vector and this block, find a comparison block to be positioned at last interpolation image or back one interpolation image, and obtain an original motion-vector and the block comparison difference of this comparison block; When this block comparison difference less than a critical value, the motion-vector that is this block with this original motion-vector of this comparison block then;
Wherein, this system of vectors of reference carries out a vector operation gained to this first, second motion-vector, by half of first motion-vector and half of second motion-vector are carried out the size that the vectorial addition computing obtains described reference vector; Deduct vector result that second motion-vector obtains according to first motion-vector and judge that whether this block is to be relevant to the image area of coverage or image appears the district, carries out oppositely to obtain this reference vector to determine the vector that whether this vectorial addition computing is produced.
6. image processor when being used for carrying out the motion picture interpolation, appearing the district to the image area of coverage/image of an interpolation image and carries out the image interpolation, and this interpolation image is made up of plurality of blocks, it is characterized in that, this device includes:
One storage element is for the original motion-vector of the relevant block that stores this interpolation image, a last interpolation image and a back one interpolation image at least; And
One computing unit is coupled to this storage element, includes:
One first candidate vector generation module, the a plurality of original motion-vector of the one original motion-vector adjacent block a plurality of with it of foundation one block, produce a numerical curve, wherein this numerical curve by to should block and a plurality of motion-vector variation values of a plurality of adjacent block formed;
Decide first motion-vector and second motion-vector of this block according to this numerical curve, wherein the motion-vector of this block is one first candidate vector, this first motion-vector system corresponds to the original motion-vector of a block of the left side paddy portion of the peak portion that is positioned at this numerical curve, and this second motion-vector system corresponds to the original motion-vector of a block of the right side paddy portion of this peak portion that is positioned at this numerical curve;
One second candidate vector generation module, this first motion-vector and second motion-vector are carried out a vector operation to obtain a reference vector of this block, wherein by half of first motion-vector and half of second motion-vector are carried out the vectorial addition computing to obtain the size of described reference vector; The direction of reference vector deducts vector result that second motion-vector obtains with first motion-vector and judges that whether this block is to be relevant to the image area of coverage or image appears the district, to determine the vector that whether this vectorial addition computing is produced to carry out oppositely to obtain this reference vector, and according to this reference vector, be that an initial position points to a comparison block in last interpolation image or back one interpolation image with this block in this interpolation image, and be one second candidate vector with the motion-vector of this comparison block in this last interpolation image or this back one interpolation image; And
One picture interpolation module comprises:
One selects module, be used for selecting first candidate vector and second candidate vector one of them, upgrade motion-vector as one, carry out the image interpolation of this block;
When wherein a block that calculates according to this second candidate vector when this comparison block in this last interpolation image or back one interpolation image is compared difference less than a certain threshold, use this second candidate vector as the motion-vector of this block; And a block that calculates according to this second candidate vector when this comparison block in this last interpolation image or back one interpolation image uses this first candidate vector as the motion-vector of this block when comparing difference and being not less than a certain threshold; The picture interpolation module determines a weighted value, and according to this first candidate vector, second candidate vector and this weighted value, carries out the image interpolation of this block according to the block comparison difference that this comparison block in last interpolation image or back one interpolation image calculates.
7. image processor as claimed in claim 6, it is characterized in that, when this block system is positioned at this image area of coverage, according to this reference vector, be this comparison block that an initial position points to last interpolation image with this block in this interpolation image, a picture interpolation module utilizes last input picture of this interpolation image to carry out the image interpolation; Appearing when district when this block is positioned at image, according to this reference vector, is this comparison block that an initial position points to back one interpolation image with this block in this interpolation image; When this block system was positioned at this image and appears the district, this picture interpolation module utilized a back input picture of this interpolation image to carry out interpolation.
8. image processor, when being used for carrying out the motion picture interpolation, original motion-vector according to relevant block in the interpolation image behind an interpolation image, the last interpolation image and, the image area of coverage/the image of this interpolation image is appeared the district carry out the image interpolation, this interpolation image is made up of plurality of blocks, it is characterized in that this device includes:
One first candidate vector generation module, be used for according to the original motion-vector of this interpolation image one block and the original motion-vector of contiguous plurality of blocks, produce one first motion-vector and one second motion-vector of this block in this interpolation image, wherein the motion-vector of this block is one first candidate vector, first motion-vector and second motion-vector of this block are determined by a numerical curve, according to this first motion-vector and this second motion-vector determines this block to be positioned at the image area of coverage and this image appears one of district, wherein this first motion-vector system corresponds to the original motion-vector of a block of the left side paddy portion of the peak portion that is positioned at this numerical curve, and this second motion-vector system corresponds to the original motion-vector of a block of the right side paddy portion of this peak portion that is positioned at this numerical curve, wherein this numerical curve is a plurality of original motion-vector according to the original motion-vector adjacent block a plurality of with it of this block, produce a numerical curve, described numerical curve by to should block and a plurality of motion-vector variation values of the horizontal block of a plurality of vicinities formed, wherein the motion-vector of this block is one first candidate vector;
One second candidate vector generation module, be used for according to this first motion-vector and this second motion-vector, calculate a reference vector of this block, wherein, computational methods are for to carry out the vectorial addition computing to half of first motion-vector and half of second motion-vector, and the vector magnitude that obtains is the size of reference vector; And for the direction of reference vector, deduct vector result that second motion-vector obtains with first motion-vector and judge that whether this block is to be relevant to the image area of coverage or image appears the district, carries out oppositely to obtain this reference vector to determine the vector that whether this vectorial addition computing is produced;
And according to this reference vector, be that an initial position points to a comparison block in last interpolation image or back one interpolation image with this block in this interpolation image, and be one second candidate vector with the motion-vector of this comparison block in this last interpolation image or this back one interpolation image; And
One picture interpolation module utilizes first motion-vector to deduct the vector result that second motion-vector obtains, judge this block be relevant to the image area of coverage or image appear the district one of them; When this interpolation block system is positioned at this image area of coverage, according to this reference vector, be this comparison block that an initial position points to last interpolation image with this block in this interpolation image, this picture interpolation module utilizes last input picture of this interpolation image to carry out the image interpolation; Appearing when district when this block is positioned at image, according to this reference vector, is this comparison block that an initial position points to back one interpolation image with this block in this interpolation image; When this interpolation block system was positioned at this image and appears the district, this picture interpolation module utilized a back input picture of this interpolation image to carry out interpolation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910000611 CN101778292B (en) | 2009-01-08 | 2009-01-08 | Image processing method and device thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910000611 CN101778292B (en) | 2009-01-08 | 2009-01-08 | Image processing method and device thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101778292A CN101778292A (en) | 2010-07-14 |
| CN101778292B true CN101778292B (en) | 2013-07-17 |
Family
ID=42514587
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910000611 Expired - Fee Related CN101778292B (en) | 2009-01-08 | 2009-01-08 | Image processing method and device thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101778292B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006012428A1 (en) * | 2004-07-21 | 2006-02-02 | Qualcomm Incorporated | Method and apparatus for motion vector assignment |
| CN1846445A (en) * | 2003-09-02 | 2006-10-11 | 皇家飞利浦电子股份有限公司 | Temporal interpolation of a pixel on basis of occlusion detection |
| CN1964462A (en) * | 2005-11-08 | 2007-05-16 | 逐点半导体（上海）有限公司 | A motion compensation frame inserting device and method to insert frame |
| EP1855474A1 (en) * | 2006-05-12 | 2007-11-14 | Sony Deutschland Gmbh | Method for generating an interpolated image between two images of an input image sequence |
| CN201444658U (en) * | 2009-03-04 | 2010-04-28 | 晨星软件研发(深圳)有限公司 | Image processing device |
-
2009
- 2009-01-08 CN CN 200910000611 patent/CN101778292B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1846445A (en) * | 2003-09-02 | 2006-10-11 | 皇家飞利浦电子股份有限公司 | Temporal interpolation of a pixel on basis of occlusion detection |
| WO2006012428A1 (en) * | 2004-07-21 | 2006-02-02 | Qualcomm Incorporated | Method and apparatus for motion vector assignment |
| CN1964462A (en) * | 2005-11-08 | 2007-05-16 | 逐点半导体（上海）有限公司 | A motion compensation frame inserting device and method to insert frame |
| EP1855474A1 (en) * | 2006-05-12 | 2007-11-14 | Sony Deutschland Gmbh | Method for generating an interpolated image between two images of an input image sequence |
| CN201444658U (en) * | 2009-03-04 | 2010-04-28 | 晨星软件研发(深圳)有限公司 | Image processing device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101778292A (en) | 2010-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102088589B (en) | Frame rate conversion using bi-directional, local and global motion estimation | |
| US8929451B2 (en) | External vectors in a motion estimation system | |
| CN106162177A (en) | Method for video coding and device | |
| CN110322542B (en) | Reconstructing views of a real world 3D scene | |
| KR100720722B1 (en) | Intermediate vector interpolation method and 3D display apparatus | |
| US8774276B2 (en) | Image processing method for determining motion vectors of interpolated picture and related apparatus | |
| CN102202224B (en) | Caption flutter-free method and apparatus used for plane video stereo transition | |
| US8447126B2 (en) | Image processing method and related apparatus | |
| KR20070069615A (en) | Motion estimator and motion estimating method | |
| WO2005079062A1 (en) | Image processing apparatus, image processing method and program | |
| JP2004530367A (en) | Motion vector prediction method and motion vector prediction device | |
| CN101692692A (en) | Method and system for electronic image stabilization | |
| CN102123244B (en) | Method and apparatus for the reparation of video stabilization | |
| CN103402045A (en) | Image de-spin and stabilization method based on subarea matching and affine model | |
| JP2012230486A (en) | Image composition apparatus, image composition method and image composition program | |
| CN109903315B (en) | Method, apparatus, device and readable storage medium for optical flow prediction | |
| CN113724155B (en) | Self-lifting learning method, device and equipment for self-supervision monocular depth estimation | |
| CN108270945B (en) | Motion compensation denoising method and device | |
| CN102065263A (en) | Image interpolation processing apparatus and method thereof | |
| Huang et al. | Algorithm and architecture design of multirate frame rate up-conversion for ultra-HD LCD systems | |
| Zhang et al. | A ViBe based moving targets edge detection algorithm and its parallel implementation | |
| CN201374785Y (en) | Image processing device for deciding motion vector of interpolated picture | |
| CN201444658U (en) | Image processing device | |
| CN107124617B (en) | Method and system for generating random vector in motion estimation motion compensation | |
| CN101778292B (en) | Image processing method and device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130717 Termination date: 20210108 |