CN103618904A - Motion estimation method and device based on pixels - Google Patents
Motion estimation method and device based on pixels Download PDFInfo
- Publication number
- CN103618904A CN103618904A CN201310586718.9A CN201310586718A CN103618904A CN 103618904 A CN103618904 A CN 103618904A CN 201310586718 A CN201310586718 A CN 201310586718A CN 103618904 A CN103618904 A CN 103618904A
- Authority
- CN
- China
- Prior art keywords
- cost
- sampling
- candidate
- width
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The embodiment of the invention discloses a motion estimation method and device based on pixels. The method comprises the steps of generating an initial price chart of an input image under each candidate vector relative to a reference image, generating a candidate price chart according to the initial price chart, choosing a candidate vector with the lowest price for the pixel of the input image according to the candidate price chart, and using the candidate vector with the lowest price as the motion vector of the pixel. The motion estimation device comprises a generating unit, a candidate unit and a selection unit. By the adoption of the motion estimation method and device based on the pixels, motion estimation based on the pixels is carried out, the motion vector based on the pixels is obtained, intensive calculation is not required, and the processing efficiency is relatively high.
Description
Technical field
The present invention relates to field of video processing, relate in particular to method for estimating and device based on pixel.
Background technology
In frame sequence image, the scenery in neighborhood graph picture frame exists certain correlation conventionally, in different picture frames, may contain identical scenery.But in different picture frames, the locus of identical scenery may be different.The locus of certain scenery in input picture,, can there is a relative displacement in the locus with respect to this scenery in reference picture, and this relative displacement is exactly the motion vector of common indication.The process of obtaining this thing motion vector is called as estimation (Motion Estimation).
Estimation is widely used a kind of technology in Video coding and Video processing.In Video coding, motion vector can be used for finding the highest image macro of correlation in reference image frame, thereby reduces the redundancy of video content.In Video coding, modal method for estimating is the estimation of rule-based macro block.The estimation of rule-based macro block, is first divided into several equal-sized macro blocks input picture conventionally; Then calculate the absolute difference sum (SAD, Sum of Absolute Difference) of each macro block correspondence under different candidate vectors; Finally choose the motion vector that candidate vector that minimum sad value is corresponding is this macro block.In Video processing, the realization of many technology need to be used motion vector equally.For example, the super-resolution technique of video deinterlacing technology, time domain, the noise reduction technology of time domain and video frame interpolation technology etc. all need to use motion vector.Different from the motion vector using in Video coding, Video processing normally be take pixel as processing unit, therefore, needs to use the motion vector based on pixel in Video processing.Motion vector based on pixel need to obtain with the method for estimating based on pixel.
Method for estimating based on pixel has multiple.Wherein more conventional method has method for estimating based on signature tracking, global optical flow method etc.Method for estimating based on signature tracking, first generates sparse optical flow; Then sparse optical flow is carried out to the intensive light stream that interpolation obtains pixel scale; According to intensive light stream, determine the motion vector of each pixel.Wherein, the process need that sparse optical flow is carried out to interpolation carries out intensive computing.Global optical flow method, need to carry out unified Modeling to the cost of global optical flow according to certain rule; Process interative computation is also optimized light stream figure, exports light stream figure corresponding to minimum cost when reaching Least-cost; Then according to light stream figure, determine the corresponding motion vector of each pixel.Wherein, interative computation process that light stream figure is optimized need to be carried out intensive computing equally.
It can be seen from the above, adopts existing method to carry out estimation, obtains the motion vector based on pixel, need to carry out the intensive computings such as interpolation or iteration, and treatment effeciency is low.
Summary of the invention
The embodiment of the present invention provides method for estimating and the device based on pixel, to solve existing method, need to carry out intensive computing, the problem that treatment effeciency is low.
First aspect, the embodiment of the present invention provides a kind of method for estimating based on pixel, comprising: generation input picture is the initial cost figure under each candidate vector with respect to reference picture; According to described initial cost figure, generate candidate's cost figure; The pixel that is described input picture according to described candidate's cost figure is chosen the candidate vector of Least-cost as the motion vector of this pixel.
In conjunction with first aspect, in the possible implementation of first aspect the first, described generation input picture is the initial cost figure under each candidate vector with respect to reference picture, comprise: adopt pixel absolute difference and the gradient absolute difference sum mode of calculating respective pixel, generation input picture is the initial cost figure under each candidate vector with respect to reference picture.
In conjunction with first aspect or the possible implementation of first aspect the first, in the possible implementation of first aspect the second, describedly according to described initial cost figure, generate candidate's cost figure, comprising: initial cost figure described in each width is smoothly obtained to level and smooth cost figure; According to described level and smooth cost figure, determine candidate's cost figure.
In conjunction with the possible implementation of first aspect the second, first aspect the third likely in implementation, described initial cost figure described in each width is smoothly obtained to level and smooth cost figure, comprising: calculate the linear relationship between initial cost figure and described input picture described in each width; According to described linear relationship, initial cost figure described in each width is smoothly obtained to level and smooth cost figure.
In conjunction with the third possible implementation of first aspect, the 4th kind of first aspect likely in implementation, before the linear relationship between initial cost figure and described input picture described in each width of described calculating, also comprise: initial cost figure described in each width is carried out to down-sampling and generate sampling cost figure; Described input picture is carried out to down-sampling and generate sampled images; Linear relationship described in each width of described calculating between initial cost figure and described input picture, comprising: calculate the described linear relationship of sampling between cost figure and described sampled images described in each width.
In conjunction with the 4th kind of possible implementation of first aspect, the 5th kind of first aspect likely in implementation, the described linear relationship of sampling between cost figure and described sampled images described in each width of obtaining, comprising: adopt linear decline method to calculate the linear relationship of sampling between cost figure and described sampled images described in each width.
In conjunction with the 4th kind of possible implementation of first aspect or the 5th kind of possible implementation of first aspect, the 6th kind of first aspect likely in implementation, describedly initial cost figure described in each width is carried out to down-sampling generate sampling cost figure, also comprise: initial cost figure described in each width is carried out to mean filter before; Describedly initial cost figure described in each width is carried out to down-sampling generate sampling cost figure, comprising: initial cost figure described in each width after mean filter is carried out to down-sampling and generate sampling cost figure.
In conjunction with the 6th kind of the 4th kind of possible implementation of first aspect, the 5th kind of possible implementation of first aspect or first aspect implementation likely, the 7th kind of first aspect likely in implementation, described described input picture is carried out before down-sampling generates sampled images, also comprising: described input picture is carried out to mean filter; Describedly described input picture is carried out to down-sampling generate sampled images, comprising: the input picture after mean filter is carried out to down-sampling and generate sampled images.
In conjunction with the 6th kind of the 4th kind of possible implementation of first aspect, the 5th kind of possible implementation of first aspect, first aspect the 7th kind of implementation or first aspect implementation likely likely, the 8th kind of first aspect likely in implementation, describedly according to described level and smooth cost figure, determine candidate's cost figure, comprising: using described level and smooth cost figure as described candidate's cost figure; Or, according to described sampling cost figure, select the preliminary election vector of Least-cost; Use described preliminary election vector to revise and obtain described candidate's cost figure level and smooth cost figure described in each width.
On the other hand, the embodiment of the present invention also provides a kind of movement estimation apparatus based on pixel, comprising: generation unit, for generate input picture with respect to reference picture the initial cost figure under each candidate vector; Candidate unit, generates candidate's cost figure for the described initial cost figure generating according to described generation unit; Selected cell, chooses the candidate vector of Least-cost as the motion vector of this pixel for the pixel that is described input picture according to described candidate's cost figure of described candidate unit generation.
In conjunction with second aspect, in the possible implementation of second aspect the first, described generation unit, for adopting pixel absolute difference and the gradient absolute difference sum mode of calculating respective pixel, generation input picture is the initial cost figure under each candidate vector with respect to reference picture.
In conjunction with second aspect or the possible implementation of second aspect the first, in the possible implementation of second aspect the second, described candidate unit comprises: level and smooth subelement, for initial cost figure described in each width is smoothly obtained to level and smooth cost figure; Determine subelement, for determining candidate's cost figure according to described level and smooth cost figure.
In conjunction with the possible implementation of second aspect the second, second aspect the third likely in implementation, described level and smooth subelement comprises: be related to computation subunit, for calculating the linear relationship between initial cost figure and described input picture described in each width; Linear smoothing subelement, for according to being related to that described linear relationship that computation subunit calculates smoothly obtains level and smooth cost figure to initial cost figure described in each width.
In conjunction with the third possible implementation of second aspect, the 4th kind of second aspect, likely in implementation, described level and smooth subelement also comprises: the first down-sampling subelement, generates sampling cost figure for initial cost figure described in each width is carried out to down-sampling; The second down-sampling subelement, generates sampled images for described input picture being carried out to down-sampling; The described computation subunit that is related to, for calculating the described linear relationship between the described sampled images of sample described in each width that described the first down-sampling subelement generates cost figure and described the second down-sampling subelement generation.
In conjunction with the 4th kind of possible implementation of second aspect, the 5th kind of second aspect likely in implementation, the described computation subunit that is related to, for adopting linear decline method to calculate to sample described in each width that described the first down-sampling subelement generates the linear relationship between the described sampled images of cost figure and described the second down-sampling subelement generation.
In conjunction with the 4th kind of possible implementation of second aspect or the 5th kind of possible implementation of second aspect, the 6th kind of second aspect likely in implementation, described level and smooth subelement also comprises: the first filtering subelement, for initial cost figure described in each width is carried out to mean filter; Described the first down-sampling subelement, carries out down-sampling for initial cost figure described in each width to after described the first filtering subelement mean filter and generates sampling cost figure.
In conjunction with the 6th kind of the 4th kind of possible implementation of second aspect, the 5th kind of possible implementation of second aspect or second aspect implementation likely, the 7th kind of second aspect likely in implementation, described level and smooth subelement also comprises: the second filtering subelement, for described input picture is carried out to mean filter; Described the second down-sampling subelement, carries out down-sampling for the input picture to after described the second filtering subelement mean filter and generates sampled images.
In conjunction with the 6th kind of the 4th kind of possible implementation of second aspect, the 5th kind of possible implementation of second aspect, second aspect the 7th kind of implementation or second aspect implementation likely likely, the 8th kind of second aspect likely in implementation, described selected cell, for using described level and smooth cost figure as described candidate's cost figure; Or, according to described sampling cost figure, select the preliminary election vector of Least-cost; Use described preliminary election vector to revise and obtain described candidate's cost figure level and smooth cost figure described in each width.
In the embodiment of the present invention, generation input picture is the initial cost figure under each candidate vector with respect to reference picture; According to described initial cost figure, generate candidate's cost figure; The pixel that is described input picture according to described candidate's cost figure is chosen the candidate vector of Least-cost as the motion vector of this pixel.Application embodiments of the invention, carry out the estimation based on pixel, obtain the motion vector based on pixel, do not need to carry out the intensive computings such as interpolation or iteration, and treatment effeciency is relatively high.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, to the accompanying drawing of required use in embodiment be briefly described below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, do not paying under the prerequisite of creative work, can also obtain according to these accompanying drawings other accompanying drawing.Shown in accompanying drawing, above-mentioned and other object of the present invention, Characteristics and advantages will be more clear.In whole accompanying drawings, identical Reference numeral is indicated identical part.Deliberately by actual size equal proportion convergent-divergent, do not draw accompanying drawing, focus on illustrating purport of the present invention.
Fig. 1 is the flow chart that the present invention is based on an embodiment of method for estimating of pixel;
Fig. 2 is the flow chart that the present invention is based on another embodiment of method for estimating of pixel;
Fig. 3 is the block diagram that the present invention is based on an embodiment of movement estimation apparatus of pixel;
Fig. 4 is the block diagram that the present invention is based on an embodiment of movement estimation apparatus candidate unit of pixel;
Fig. 5 A is the block diagram that the present invention is based on an embodiment of the level and smooth subelement of movement estimation apparatus of pixel;
Fig. 5 B is the block diagram that the present invention is based on an embodiment of the level and smooth subelement of movement estimation apparatus of pixel;
Fig. 5 C is the block diagram that the present invention is based on an embodiment of the level and smooth subelement of movement estimation apparatus of pixel.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out to clear, complete description, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Embodiment based in the present invention, those of ordinary skills, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.
Referring to Fig. 1, for the present invention is based on the flow chart of an embodiment of method for estimating of pixel.This embodiment comprises the steps:
For speed up processing, and keep the consistency of final result, when the estimation of carrying out based on pixel, can choose in advance N vector as candidate vector, wherein N >=2; Then from this N candidate vector, for each pixel of input picture, select a candidate vector of Least-cost, as the motion vector of this pixel.Now, the process of estimation is the process of choosing motion vector from candidate vector for each pixel.The mode of choosing of candidate vector has multiple, for example, can be using the motion vector of processed image pixel as candidate vector, wherein, processed image can be the image coexisting in an image sequence with input picture.
When choosing motion vector, first need calculating input image with respect to reference picture the initial cost figure under each candidate vector.Each pixel is with respect to the respective pixel in reference picture in input picture for initial cost figure, and the cost under candidate vector forms.If there be N candidate vector, can obtain so the initial cost figure of N width, N candidate vector is corresponding one by one with the initial cost figure of N width.
After getting initial cost figure, can first generate the level and smooth cost figure that the initial cost figure of each width is corresponding; Then according to level and smooth cost figure, determine the candidate cost figure corresponding with the initial cost figure of each width.If there is the initial cost figure of N width, correspondingly can generate the level and smooth cost figure of N width, figure is corresponding one by one with the initial cost figure of N width for N width candidate cost.
While generating level and smooth cost figure, can adopt directly level and smooth mode.Specifically, can, according to the linear relationship between initial cost figure and input picture, initial cost figure be carried out smoothly, obtaining level and smooth cost figure.Initial cost figure is carried out smoothly, can making choosing of motion vector more accurate, and there is good consistency.Linear relationship between initial cost figure and input picture, can adopt linear decline method to calculate.
While generating level and smooth cost figure, also can adopt the rear level and smooth mode of first sampling.Specifically, can be first initial cost figure be carried out to the down-sampling cost figure that obtains sampling, and input picture is carried out to down-sampling obtain sampled images; Then according to the linear relationship between sampling cost figure and sampled images, to sampling cost, figure carries out smoothly, generating level and smooth cost figure.Initial cost figure and input picture are carried out to down-sampling, can, guaranteeing that motion vector chooses accurately under prerequisite, reduce computation complexity.Linear relationship between sampling cost figure and sampled images, also can adopt linear decline method to calculate.
Before initial cost figure is sampled, can also first to initial cost figure, carry out mean filter; Same, before input picture is sampled, also can first to input picture, sample and carry out mean filter.First initial cost figure and input picture are carried out to mean filter, and then sample, can increase the robustness of processing procedure, reduce the impact of picture noise.
According to level and smooth cost figure, determine that candidate's cost figure also can have various ways.For example, can be after getting level and smooth cost figure, directly using level and smooth cost figure as candidate's cost figure; Or, also can for preliminary election vector of each pixel selection, then use preliminary election vector to revise level and smooth cost figure first according to initial cost figure, using revised level and smooth cost figure as candidate's cost figure, thereby improve the selected probability of preliminary election vector.
When initial cost figure is N width, to the initial cost figure of each width, can adopt preceding method to carry out smoothly.
From the aforementioned process of obtaining candidate's cost figure, according to N the N width candidate cost figure that candidate vector is corresponding, can know that pixel j is with respect to pixel mi+j, cost under each candidate vector, wherein, pixel j is the pixel in input picture, and pixel mi+j is pixel corresponding with pixel j in reference picture; Thereby can be for the candidate vector of a Least-cost of each pixel selection in input picture I be as motion vector.When choosing motion vector for each pixel, can entirely take (WTA, Winner Take All) rule according to victor, the concrete process of choosing just repeats no more at this.
In the embodiment of the present invention, generation input picture is the initial cost figure under each candidate vector with respect to reference picture; According to described initial cost figure, generate candidate's cost figure; The pixel that is described input picture according to described candidate's cost figure is chosen the candidate vector of Least-cost as the motion vector of this pixel.Application the present embodiment, carries out the estimation based on pixel, obtains the motion vector based on pixel, does not need to carry out intensive computing, and treatment effeciency is relatively high.
Referring to Fig. 2, for the present invention is based on the flow chart of another embodiment of method for estimating of pixel, this embodiment describes the method for estimating that the present invention is based on pixel in detail.
Step 201, adopts pixel absolute difference and the gradient absolute difference sum mode of calculating respective pixel, and generation input picture is the initial cost figure under each candidate vector with respect to reference picture.
For ease of describing, input picture can be designated as I, and reference picture can be designated as R, and candidate vector can be designated as i, and initial cost figure can be designated as C.The pixel of input picture I can be designated as j, and pixel corresponding with pixel j in reference image R can be designated as mi+j, and input picture I can be designated as C with respect to reference image R initial cost figure under candidate vector i
i.Pixel j in input picture I, with respect to pixel mi+j corresponding with pixel j in reference image R, the initial cost under candidate vector i can be referred to as the initial cost of pixel j.
Initial cost figure C
ican in input picture I, the cost of each pixel form.The initial cost of pixel j can be used pixel j and the pixel absolute difference of pixel mi+j and the gradient absolute difference computation of pixel j and pixel mi+j.Under candidate vector i, the initial cost C of pixel j
i,j=| I
j-R
mi+j|+| D
j-D
mi+j|, wherein | I
j-R
mi+j| be pixel j and pixel mi+j pixel absolute difference, | D
j-D
mi+j| be pixel j and pixel mi+j gradient absolute difference.Because each pixel can represent by the coordinate figure in image by it, therefore, the pixel absolute difference of pixel j and pixel mi+j can be calculated according to the coordinate figure of pixel j and pixel mi+j coordinate figure, the gradient absolute difference of pixel j and pixel mi+j can utilize the Grad of pixel j and pixel mi+j Grad to calculate, and concrete computational process just repeats no more at this.
When candidate vector has when a plurality of, can adopt said method to obtain respectively the initial cost figure corresponding with each candidate vector.Adopt pixel absolute difference and gradient absolute difference sum to calculate the cost of each pixel, computational process is simple and can in cost figure, retain the local edge of input picture.
Step 202, carries out mean filter to initial cost figure described in each width.
Mean filter is a kind of typical linear filtering algorithm, it refers on image sets a Filtering Template for object pixel, and this template comprises the adjacent pixels around object pixel, for example, 8 pixels of surrounding centered by object pixel, form a Filtering Template; With the mean value of all pixels in template, replace original pixel value again, the mean value here and pixel value can be color-values, gray value etc.The concrete methods of realizing of mean filter has a lot, at this, just repeats no more.Initial cost figure is carried out to mean filter, can reduce picture noise and motion vector be chosen to the impact of result.
Step 203, carries out the down-sampling cost figure that obtains sampling to initial cost figure described in each width after mean filter.
For reducing computational process complexity, can carry out down-sampling to the initial cost figure after mean filter.The ratio of down-sampling can be set as required.For example, can carry out four samplings to initial cost figure horizontal direction, vertical direction is carried out three samplings.If there is the initial cost figure of N width, respectively the initial cost figure of each width is carried out to down-sampling, generate the sampling cost figure that the initial cost figure of each width is corresponding.
Step 204, carries out mean filter to described input picture.
Input picture is carried out to mean filter, can on motion vector, choose the impact of result by noise decrease.
Step 205, carries out down-sampling to the described input picture after mean filter and obtains sampled images.
For reducing computational process complexity, can also carry out down-sampling to the input picture after mean filter.The ratio of down-sampling can be identical with the ratio of initial cost figure being carried out to down-sampling.
At this, it should be noted that, can first perform step 202 to step 203, then perform step 204 to step 205; Also can first perform step 204 to step 205, then perform step 202 to step 203, this present invention is not limited.
Step 206, calculates the described linear relationship of sampling between cost figure and described sampled images described in each width;
Linear relationship between sampling cost figure and sampled images, the linear relationship in sampled images between each pixel cost corresponding with this pixel in the cost figure that samples forms.Linear relationship between the pixel cost corresponding with this pixel, can calculate according to the pixel intensity cost corresponding with this pixel.When having N width sampling cost figure, can adopt linear decline method to obtain one by one the linear relationship between each width sampling cost figure and sampled images.
Linear relationship in sampled images between the pixel cost corresponding with this pixel in sampling cost figure, is determined by the linear relationship coefficient between the two, as long as get the linear relationship coefficient between the two, just can determine the linear relationship between the two.When calculating the linear relationship coefficient of certain pixel, first for this pixel, on sampled images, set a partial image window, in partial image window, comprise this pixel.The size of topography can be set according to demand.For example, can adopt the image block of 13*13 as partial image window.Computational process can comprise the steps: to calculate the mean flow rate of each pixel in partial image window
and the standard variance var (I ' I ') of brightness, wherein
calculating under candidate vector i, the average cost value of pixel in partial image window
and the covariance cov (C of pixel cost value in partial image window
i' I '), wherein
according to above-mentioned mean flow rate
the standard variance var (I ' I ') of brightness, average cost value
and the covariance cov (C of cost value
i' I'), can calculate this pixel and calculate linear relationship coefficient.Linear relationship coefficient comprises slope a
iwith intercept b
i, wherein,
as shown in aforementioned formula, calculate standard variance and covariance can be with the average computing of partial image window.The technology such as use integrogram can be so that computational complexity and window size be irrelevant.The method of calculating linear relationship coefficient is multiple in addition, introduces no longer in detail here.
If there is N width sampling cost figure, can adopt preceding method, obtain respectively the linear relationship between each width sampling cost figure and input picture, obtain N the linear relationship one to one with N width sampling cost figure.Adopt the method for linear decline to calculate the linear relationship between down-sampled images brightness and sampling cost figure, can only carry out single pass, thereby reduce computational complexity.
Step 207, smoothly obtains level and smooth cost figure according to described linear relationship to initial cost figure described in each width.
After the linear relationship getting between sampling cost figure and sampled images, can be according to described linear relationship, to sampling cost, figure carries out smoothly.With sampling cost figure C
i' corresponding level and smooth cost figure can be designated as C
i".To sampling cost figure C
i' carry out smoothly, can pass through the C to sampling cost figure
i' the cost of each pixel of comprising smoothly realizes.For example, under candidate vector i, pixel j in input picture I, in the sampled images corresponding with input picture, pixel k after the sampling corresponding with pixel j, the cost that pixel j is after level and smooth can be designated as C "
ik, known C "
ik=a
ik* I
j+ b
ik, a wherein
ikfor the slope corresponding with pixel j, b
ikfor the intercept corresponding with pixel j, I
jbrightness for pixel j.
Step 208, selects the preliminary election vector of Least-cost according to described sampling cost figure.
Being carried out to down-sampling, initial cost figure described in each width obtains sampling after cost figure, can be according to sampling cost figure, and for each pixel is chosen the motion vector of a Least-cost in advance, as preliminary election vector.When choosing preliminary election vector for each pixel, can entirely take (WTA, Winner Take All) rule according to victor, the concrete process of choosing just repeats no more at this.
Step 209, is used described preliminary election vector to revise level and smooth cost figure described in each width, generates candidate's cost figure.
After getting preliminary election vector, can to level and smooth cost figure, revise according to preliminary election vector, generate candidate's cost figure, improve the selected probability of preliminary election vector.For example, if in sampled images, be t with the preliminary election vector of the pixel k that in input picture I, pixel j is corresponding.The level and smooth cost figure C that preliminary election vector t is corresponding
t" in, the corresponding cost of pixel k can be expressed as C
tk", can be by C
tk" be multiplied by the coefficient that is less than 1, thereby improve the probability that motion vector t becomes pixel j Least-cost candidate vector in input picture I.Reference when preliminary election vector is selected as vector, in texture rareness or the larger input picture of noise, can improve the consistency of motion vector.
Step 210, the pixel that is described input picture according to described candidate's cost figure is chosen the candidate vector of Least-cost as the motion vector of this pixel.
After getting candidate's cost figure, according to WTA rule, be the motion vector of a Least-cost of each pixel selection.
At this, it should be noted that, after execution of step 207, can directly perform step 210, that is, after getting level and smooth cost figure, can level and smooth cost figure not revised, directly using level and smooth cost figure as candidate's cost figure; Then be the motion vector of a Least-cost of each pixel selection.Directly, using level and smooth cost figure as candidate's cost figure, can simplify processing procedure, reduce data amount of calculation.
From above-described embodiment, can find out, Application Example is carried out the estimation based on pixel, not only calculate simple, treatment effeciency is relatively high, be applicable to using hardware to realize, and can accurately choose motion vector, and make the motion vector of choosing have reasonable consistency, also have good robustness simultaneously.
Corresponding with the method for estimating that the present invention is based on pixel, the present invention also provides the movement estimation apparatus based on pixel.
Referring to Fig. 3, for the present invention is based on embodiment block diagram of movement estimation apparatus of pixel.
This device comprises: generation unit 301, candidate unit 302, selected cell 303.
Wherein, described generation unit 301, for generate input picture with respect to reference picture the initial cost figure under each candidate vector.
The concrete mode that generation unit 301 generates input picture initial cost figure under each candidate vector with respect to reference picture can have multiple.For example, generation unit 301 can be for adopting pixel absolute difference and the gradient absolute difference sum mode of calculating respective pixel, and generation input picture is the initial cost figure under each candidate vector with respect to reference picture.Concrete generative process just repeats no more at this.
Described candidate unit 302, generates candidate's cost figure for the described initial cost figure generating according to described generation unit 301.
At generation unit 301, generate after initial cost figure, first candidate unit 302 can generate the level and smooth cost figure that the initial cost figure of each width is corresponding; Then according to level and smooth cost figure, determine the candidate cost figure corresponding with the initial cost figure of each width.
As shown in Figure 4, candidate unit 302 can comprise: level and smooth subelement 401 and definite subelement 402.Level and smooth subelement 401 is for smoothly obtaining level and smooth cost figure to initial cost figure described in each width; Determine that subelement 402 is for determining candidate's cost figure according to described level and smooth cost figure.
As shown in Figure 5A, level and smooth subelement 401 can comprise: be related to computation subunit 501, linear smoothing subelement 502.Wherein, the described computation subunit 501 that is related to, for calculating the linear relationship between initial cost figure and described input picture described in each width; Described linear smoothing subelement 502, smoothly obtains level and smooth cost figure for the described linear relationship calculating according to computation subunit 501 to initial cost figure described in each width.
For reducing data amount of calculation, speed up processing, as shown in Figure 5 B, level and smooth subelement 401 can also comprise: the first down-sampling subelement 503 and the second down-sampling subelement 504.Wherein, the first down-sampling subelement 503, generates sampling cost figure for initial cost figure described in each width is carried out to down-sampling; The second down-sampling subelement 504, generates sampled images for described input picture being carried out to down-sampling.When level and smooth subelement 401 comprises the first down-sampling subelement 503 and the second down-sampling subelement 504, be related to computation subunit 501, can be for calculating the described linear relationship between the described sampled images of sample described in each width that described the first down-sampling subelement 503 generates cost figure and described the second down-sampling subelement 504 generations.
For reducing the impact of noise in image point, as shown in Figure 5 C, level and smooth subelement 401 can also comprise: the first filtering subelement 505 and the second filtering subelement 506.Wherein, the first filtering subelement 505, for carrying out mean filter to initial cost figure described in each width; The second filtering subelement 506, for carrying out mean filter to described input picture.When level and smooth subelement comprises the first filtering subelement 505, described the first down-sampling subelement 503, can carry out down-sampling for initial cost figure described in each width to after described the first filtering subelement 505 mean filters and generate sampling cost figure.When level and smooth subelement comprises the second filtering subelement 506, described the second down-sampling subelement 504, can carry out for the input picture to after described the second filtering subelement 506 mean filters down-sampling and generate sampled images.
If described level and smooth subelement 401 comprises described the first down-sampling subelement 503, described definite subelement 402 can be selected for the described sampling cost figure generating according to described the first down-sampling subelement 503 the preliminary election vector of Least-cost; Then use described preliminary election vector to revise and obtain described candidate's cost figure level and smooth cost figure described in each width.
Described selected cell 303, chooses the candidate vector of Least-cost as the motion vector of this pixel for the pixel that is described input picture according to described candidate's cost figure of described candidate unit 302 generations.
From above-described embodiment, can find out, the movement estimation apparatus based on pixel that adopts the present embodiment to provide, carries out the estimation based on pixel, obtains the motion vector based on pixel, does not need to carry out intensive computing, and treatment effeciency is relatively high.Because computational process is simple, do not need interpolation or multiple scanning, be applicable to hardware and realize.
Those skilled in the art can be well understood to the mode that technology in the embodiment of the present invention can add essential general hardware platform by software and realize.Understanding based on such, the part that technical scheme in the embodiment of the present invention contributes to prior art in essence in other words can embody with the form of software product, this computer software product can be stored in storage medium, as ROM/RAM, magnetic disc, CD etc., comprise that some instructions are with so that a computer equipment (can be personal computer, server, or the network equipment etc.) carry out the method described in some part of each embodiment of the present invention or embodiment.
Each embodiment in this specification all adopts the mode of going forward one by one to describe, between each embodiment identical similar part mutually referring to, each embodiment stresses is the difference with other embodiment.Especially, for system embodiment, because it is substantially similar in appearance to embodiment of the method, so description is fairly simple, relevant part is referring to the part explanation of embodiment of the method.
Above-described embodiment of the present invention, does not form limiting the scope of the present invention.Any modification of doing within the spirit and principles in the present invention, be equal to and replace and improvement etc., within all should being included in protection scope of the present invention.
Claims (18)
1. the method for estimating based on pixel, is characterized in that, comprising:
Generation input picture is the initial cost figure under each candidate vector with respect to reference picture;
According to described initial cost figure, generate candidate's cost figure;
The pixel that is described input picture according to described candidate's cost figure is chosen the candidate vector of Least-cost as the motion vector of this pixel.
2. the method for claim 1, is characterized in that, described generation input picture is the initial cost figure under each candidate vector with respect to reference picture, comprising:
Adopt pixel absolute difference and the gradient absolute difference sum mode of calculating respective pixel, generation input picture is the initial cost figure under each candidate vector with respect to reference picture.
3. method as claimed in claim 1 or 2, is characterized in that, describedly according to described initial cost figure, generates candidate's cost figure, comprising:
Initial cost figure described in each width is smoothly obtained to level and smooth cost figure;
According to described level and smooth cost figure, determine candidate's cost figure.
4. method as claimed in claim 3, is characterized in that, described initial cost figure described in each width is smoothly obtained to level and smooth cost figure, comprising:
Calculate the linear relationship between initial cost figure and described input picture described in each width;
According to described linear relationship, initial cost figure described in each width is smoothly obtained to level and smooth cost figure.
5. method as claimed in claim 4, is characterized in that, before the linear relationship between initial cost figure and described input picture described in each width of described calculating, also comprises:
Initial cost figure described in each width is carried out to down-sampling and generate sampling cost figure;
Described input picture is carried out to down-sampling and generate sampled images;
Linear relationship described in each width of described calculating between initial cost figure and described input picture, comprising:
Calculate the described linear relationship of sampling between cost figure and described sampled images described in each width.
6. method as claimed in claim 5, is characterized in that, described in obtain the linear relationship of sampling between cost figure and described sampled images described in each width, comprising:
Adopt linear decline method to calculate the linear relationship of sampling between cost figure and described sampled images described in each width.
7. the method as described in claim 5 or 6, is characterized in that, describedly initial cost figure described in each width is carried out to down-sampling generates sampling cost figure, also comprises before:
Initial cost figure described in each width is carried out to mean filter;
Describedly initial cost figure described in each width carried out to down-sampling generate sampling cost figure, comprising:
Initial cost figure described in each width after mean filter is carried out to down-sampling and generate sampling cost figure.
8. the method as described in claim 5 to 7 any one claim, is characterized in that, described described input picture is carried out before down-sampling generates sampled images, also comprising:
Described input picture is carried out to mean filter;
Describedly described input picture carried out to down-sampling generate sampled images, comprising:
Input picture after mean filter is carried out to down-sampling and generate sampled images.
9. the method as described in claim 5 to 8 any one claim, is characterized in that, describedly according to described level and smooth cost figure, determines candidate's cost figure, comprising:
Using described level and smooth cost figure as described candidate's cost figure; Or,
According to described sampling cost figure, select the preliminary election vector of Least-cost; Use described preliminary election vector to revise and obtain described candidate's cost figure level and smooth cost figure described in each width.
10. the movement estimation apparatus based on pixel, is characterized in that, comprising:
Generation unit, for generate input picture with respect to reference picture the initial cost figure under each candidate vector;
Candidate unit, generates candidate's cost figure for the described initial cost figure generating according to described generation unit;
Selected cell, chooses the candidate vector of Least-cost as the motion vector of this pixel for the pixel that is described input picture according to described candidate's cost figure of described candidate unit generation.
11. devices as claimed in claim 10, is characterized in that,
Described generation unit, for adopting pixel absolute difference and the gradient absolute difference sum mode of calculating respective pixel, generation input picture is the initial cost figure under each candidate vector with respect to reference picture.
12. devices as described in claim 10 or 11, is characterized in that, described candidate unit comprises:
Level and smooth subelement, for smoothly obtaining level and smooth cost figure to initial cost figure described in each width;
Determine subelement, for determining candidate's cost figure according to described level and smooth cost figure.
13. devices as claimed in claim 12, is characterized in that, described level and smooth subelement comprises:
Be related to computation subunit, for calculating the linear relationship between initial cost figure and described input picture described in each width;
Linear smoothing subelement, for according to being related to that described linear relationship that computation subunit calculates smoothly obtains level and smooth cost figure to initial cost figure described in each width.
14. devices as claimed in claim 13, is characterized in that, described level and smooth subelement also comprises:
The first down-sampling subelement, generates sampling cost figure for initial cost figure described in each width is carried out to down-sampling;
The second down-sampling subelement, generates sampled images for described input picture being carried out to down-sampling;
The described computation subunit that is related to, for calculating the described linear relationship between the described sampled images of sample described in each width that described the first down-sampling subelement generates cost figure and described the second down-sampling subelement generation.
15. devices as claimed in claim 14, is characterized in that,
The described computation subunit that is related to, for adopting linear decline method to calculate to sample described in each width that described the first down-sampling subelement generates the linear relationship between the described sampled images of cost figure and described the second down-sampling subelement generation.
16. devices as described in claims 14 or 15, is characterized in that, described level and smooth subelement also comprises:
The first filtering subelement, for carrying out mean filter to initial cost figure described in each width;
Described the first down-sampling subelement, carries out down-sampling for initial cost figure described in each width to after described the first filtering subelement mean filter and generates sampling cost figure.
17. devices as described in claim 14 to 16 any one claim, is characterized in that, described level and smooth subelement also comprises:
The second filtering subelement, for carrying out mean filter to described input picture;
Described the second down-sampling subelement, carries out down-sampling for the input picture to after described the second filtering subelement mean filter and generates sampled images.
18. devices as described in claim 14 to 17 any one claim, is characterized in that,
Described selected cell, for using described level and smooth cost figure as described candidate's cost figure; Or,
According to described sampling cost figure, select the preliminary election vector of Least-cost; Use described preliminary election vector to revise and obtain described candidate's cost figure level and smooth cost figure described in each width.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310586718.9A CN103618904B (en) | 2013-11-20 | 2013-11-20 | Motion estimation method and device based on pixels |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310586718.9A CN103618904B (en) | 2013-11-20 | 2013-11-20 | Motion estimation method and device based on pixels |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103618904A true CN103618904A (en) | 2014-03-05 |
| CN103618904B CN103618904B (en) | 2017-02-22 |
Family
ID=50169608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310586718.9A Active CN103618904B (en) | 2013-11-20 | 2013-11-20 | Motion estimation method and device based on pixels |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103618904B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106027851A (en) * | 2015-03-30 | 2016-10-12 | 想象技术有限公司 | Image filtering based on image gradients |
| WO2017036399A1 (en) * | 2015-09-02 | 2017-03-09 | Mediatek Inc. | Method and apparatus of motion compensation for video coding based on bi prediction optical flow techniques |
| TWI663872B (en) * | 2017-03-16 | 2019-06-21 | 聯發科技股份有限公司 | Method and apparatus of motion refinement based on bi-directional optical flow for video coding |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5471252A (en) * | 1993-11-03 | 1995-11-28 | Matsushita Electric Industrial Corporation Of America | Method and apparatus for estimating motion vector fields by rejecting local outliers |
| US5825909A (en) * | 1996-02-29 | 1998-10-20 | Eastman Kodak Company | Automated method and system for image segmentation in digital radiographic images |
| EP1841232A1 (en) * | 2006-03-31 | 2007-10-03 | Sony Deutschland Gmbh | Method and apparatus to improve the convergence speed of a recursive motion estimator |
| EP2227012A1 (en) * | 2009-03-05 | 2010-09-08 | Sony Corporation | Method and system for providing reliable motion vectors |
| CN101629966B (en) * | 2009-08-18 | 2011-09-28 | 清华大学深圳研究生院 | Particle image velocimetry (PIV) processing method |
| US8711938B2 (en) * | 2009-09-04 | 2014-04-29 | Sharp Laboratories Of America, Inc. | Methods and systems for motion estimation with nonlinear motion-field smoothing |
| JP2011233039A (en) * | 2010-04-28 | 2011-11-17 | Sony Corp | Image processor, image processing method, imaging device, and program |
| CN102819764B (en) * | 2012-07-18 | 2015-07-08 | 郑州金惠计算机系统工程有限公司 | Method for counting pedestrian flow from multiple views under complex scene of traffic junction |
| CN103247058B (en) * | 2013-05-13 | 2015-08-19 | 北京工业大学 | A kind of quick the Computation of Optical Flow based on error Distributed-tier grid |
-
2013
- 2013-11-20 CN CN201310586718.9A patent/CN103618904B/en active Active
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106027851A (en) * | 2015-03-30 | 2016-10-12 | 想象技术有限公司 | Image filtering based on image gradients |
| CN106027851B (en) * | 2015-03-30 | 2021-02-02 | 想象技术有限公司 | Method and system for processing images |
| US11244432B2 (en) | 2015-03-30 | 2022-02-08 | Imagination Technologies Limited | Image filtering based on image gradients |
| WO2017036399A1 (en) * | 2015-09-02 | 2017-03-09 | Mediatek Inc. | Method and apparatus of motion compensation for video coding based on bi prediction optical flow techniques |
| CN107925775A (en) * | 2015-09-02 | 2018-04-17 | 联发科技股份有限公司 | The motion compensation process and device of coding and decoding video based on bi-directional predicted optic flow technique |
| TWI663872B (en) * | 2017-03-16 | 2019-06-21 | 聯發科技股份有限公司 | Method and apparatus of motion refinement based on bi-directional optical flow for video coding |
| US11109062B2 (en) | 2017-03-16 | 2021-08-31 | Mediatek Inc. | Method and apparatus of motion refinement based on bi-directional optical flow for video coding |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103618904B (en) | 2017-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11595653B2 (en) | Processing of motion information in multidimensional signals through motion zones and auxiliary information through auxiliary zones | |
| CN105517671B (en) | Video frame interpolation method and system based on optical flow method | |
| US9767598B2 (en) | Smoothing and robust normal estimation for 3D point clouds | |
| KR101634562B1 (en) | Method for producing high definition video from low definition video | |
| JP5157768B2 (en) | Image processing apparatus and method, and program | |
| EP2164040B1 (en) | System and method for high quality image and video upscaling | |
| TWI739151B (en) | Method, device and electronic equipment for image generation network training and image processing | |
| US8204124B2 (en) | Image processing apparatus, method thereof, and program | |
| TW201146011A (en) | Bi-directional, local and global motion estimation based frame rate conversion | |
| CN105409196A (en) | Adaptive path smoothing for video stabilization | |
| WO2014045026A1 (en) | Systems and methods for reducing noise in video streams | |
| JP2010148037A (en) | Moving image playback apparatus, moving image playback method and moving image playback program | |
| KR100860968B1 (en) | Image-resolution-improvement apparatus and method | |
| CN114648451A (en) | Intelligent noise reduction | |
| WO2013089261A1 (en) | Image processing system, and image processing method | |
| CN103618904A (en) | Motion estimation method and device based on pixels | |
| Hou et al. | Joint learning of image deblurring and depth estimation through adversarial multi-task network | |
| CN112801883A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN114266850A (en) | Point cloud completion method based on continuous standardized stream and anti-refinement network | |
| CN115631223A (en) | Multi-view stereo reconstruction method based on self-adaptive learning and aggregation | |
| Smolka et al. | Impulsive noise suppression in color images based on the geodesic digital paths | |
| Tolstaya et al. | Depth propagation for semi-automatic 2d to 3d conversion | |
| Chiu et al. | Voting-based depth map refinement and propagation for 2D to 3D conversion | |
| Mahmood et al. | Measuring focus quality in vector valued images for shape from focus | |
| JP5934019B2 (en) | Gradation restoration device and program thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |