CN102238380B - Hierarchical motion estimation method and system - Google Patents
Hierarchical motion estimation method and system Download PDFInfo
- Publication number
- CN102238380B CN102238380B CN 201010168223 CN201010168223A CN102238380B CN 102238380 B CN102238380 B CN 102238380B CN 201010168223 CN201010168223 CN 201010168223 CN 201010168223 A CN201010168223 A CN 201010168223A CN 102238380 B CN102238380 B CN 102238380B
- Authority
- CN
- China
- Prior art keywords
- scan line
- sampling
- motion estimation
- present frame
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention discloses a hierarchical motion estimation method and a hierarchical motion estimation system. The method comprises the following steps of: down-sampling on a reference frame and a current frame; storing the down-sampled reference frame; generating a rough motion vector map (MV map) according to the down-sampled reference frame and the down-sampled current frame; acquiring and storing a scanning line adjacent to a central scanning line which corresponds to a down-sampled scanning line of the down-sampled reference frame; and generating a detailed motion vector (MV) map according to the rough MV map, the current frame and the stored scanning line adjacent to the central scanning line.
Description
Technical field
The relevant image of the present invention is handled, particularly about a kind of hierarchical motion estimation (hierarchicalmotion estimation).
Background technology
(motion estimative, ME) (motionvector in the time of MV), need capture the pixel data of reference frame (for example former frame) from external memory devices to produce motion vector carrying out estimation.Yet, be subject to the transmission bandwidth of storage arrangement, pixel data also can't directly be extracted by storage arrangement (for example double data speed synchronous dynamic RAM (double data ratesynchronous dynamic random access memory, DDR SDRAM)) immediately.
In order to address the above problem, can use the internal storage (for example high-speed cache) of integrated circuit, with the some (for example search area) of temporary transient storage reference frame.Yet, resolve (high-definition is hereinafter to be referred as HD) image (its resolution can be 1920x1080) for height, the capacity of internal storage produces not enough again.For example, during as search area, need the memory span of 108 (also namely, 1080* (1/10)) bar scan line with 1/10 size of HD image, or (108*1920*8) the capacity of position also namely, that is equivalent to 1658880.
In view of traditional movement estimation system or method and can't be applicable to the image of high-resolution effectively, therefore, need the mechanism that proposes a kind of novelty badly, resolve higher image, for example HD image to be applicable to.
Summary of the invention
In view of above-mentioned, one of purpose of the embodiment of the invention is to provide a kind of hierarchical motion estimation System and method for, and it is externally under the limiting bandwidth of memory, reduces the demand of internal storage and can not influence the precision of estimation.
According to the embodiment of the invention, first reduces sampling unit reduces the sampling reference frame, and the second reduction sampling unit reduction sampling present frame, wherein reduces the sampling reference frame and is stored in rough line buffer.Coarse movement vector (MV) estimator is according to reducing the sampling reference frame and reducing the sampling present frame, to produce the coarse movement vectogram.The acquisition of fine lines buffer and storage are adjacent to the scan line of central scan line, and wherein this central scan line is corresponding to the reduction sampling scan line that reduces the sampling reference frame.Fine movement vector estimator estimates that to produce fine movement (MV) schemes according to coarse movement vector (MV) figure, present frame and storage scan line adjacent to the central scan line.
Description of drawings
The calcspar of Fig. 1 shows hierarchical motion estimation (ME) system of the embodiment of the invention.
The flow chart of Fig. 2 shows the layering motion estimation method of the embodiment of the invention.
Fig. 3 shows a part that reduces the sampling frame.
Group's estimation of Fig. 4 view similar.
[main element symbol description]
10 first reduce sampling unit
11 second reduce sampling unit
12 rough line buffers
13 coarse movement vector (MV) estimators
14 fine lines buffers
15 fine movement vector (MV) estimators
The 21-25 step
Embodiment
The calcspar of Fig. 1 shows hierarchical motion estimation (ME) system of the embodiment of the invention.The flow chart of Fig. 2 shows the layering motion estimation method of the embodiment of the invention.The embodiment of the invention is resolved the coding of image (for example resolution is 1920x1080) applicable to height, but is not limited to this.Though present embodiment shows hierarchical motion estimation (ME) method of two-stage, the embodiment of the invention is also applicable to the hierarchical motion estimation more than the two-stage (ME).
In the phase I of the hierarchical motion estimation (ME) of present embodiment, in step 21, reduce by 10 pairs of former frame of sampling unit (being generally reference frame) with first and reduce sampling (or sub-sampling).Generally speaking, use the height that reduces the frame of sampling factor N to do to reduce sampling, and use the width that reduces the frame of sampling factor M to do to reduce sampling.Present embodiment then is to adopt the take a sample height and the width of frame of identical reduction sampling factor N.In a specific embodiment, select the search area of former frame to do to reduce sampling.This search area is the part (for example account for primitive frame 1/10) of primitive frame.Fig. 3 shows that using a part that reduces by 4 pairs of frames of sampling factor to do to reduce takes a sample.In this example, on the level and vertical direction of the search area of frame, per 4 pixels are chosen a pixel.Therefore, the capacity of data will be reduced to 1/16 ((1/4) * (1/4), or be generally (1/N) * (1/M) also namely) of original search area.In same step, then use second to reduce sampling unit 11 to reduce sampling for the present frame that is about to encode.Next, in step 22, the former frame that reduces sampling is stored in rough line buffer (coarse linebuffer) 12.For instance, if the search area of high parse for frame comprises 108 scan lines, then the former frame that reduces sampling can be stored in the rough line buffer 12 with 108* (1/4) * (1/4) capacity.Be noted that though (forward) forward estimation that present embodiment adopts uses former frame as the reference frame, present embodiment also can be applicable to backward (backward) estimation, it uses back one frame as the reference frame.
Next, in step 23, rough (coarse) motion vector (MV) estimator 13 produces coarse movement vectogram (MV map) according to reducing the former frame of sampling with the present frame that reduces sampling.The coarse movement vectogram that produces shows that present frame is corresponding to motion or the displacement of former frame (or reference frame).Wherein, for for the estimation of block, each macro zone block in motion vector figure comprises a motion vector (the MV) (horizontal component that includes motion vector, the vertical component of motion vector), with motion or the displacement of the macro zone block in the expression present frame corresponding to the macro zone block in the former frame.Can use traditional tolerance, for example (but not being defined as) absolute difference and (sum of absolute differences, SAD), to produce the coarse movement vector.
Second stage about the hierarchical motion estimation (ME) of present embodiment, in step 24, adjacent scan line (can be captured by external memory storage with reducing the sampling scan line in the acquisition former frame, double data speed synchronous dynamic RAM (DDR SDRAM) for example), and with the scan line that captures be stored in fine lines buffer (refine line buffer) 14.In the present embodiment, if the reduction sampling factor of height is N, then captures and reduced sampling scan line (being also referred to as the central scan line) upwards and downward each N bar scan line, and together with the central scan line.In other words, store (2*N+1) bar scan line altogether in fine lines buffer 14.Fig. 3 shows (2*4+1) bar adjacent scanning lines when N=4.
Next, in step 25, meticulous (refine) motion vector (MV) estimator 15 is according to coarse movement vector (MV) figure, present frame and be stored in the scan line of fine lines buffer 14, to produce fine movement vector (MV) figure.By this, the accuracy of the motion vector (MV) that coarse movement vector (MV) estimator 13 can be produced is meticulous by the N pixel is 1 pixel.Can use traditional tolerance, (but not being defined as) absolute difference and (SAD) for example is to produce fine movement vector (MV).
In present frame, the vertical component of the motion vector of adjacent macro zone block (MV) is normally different, make corresponding to respectively the organizing adjacent scanning lines and must be loaded into fine lines buffer 14 again of different vertical component, thereby cause the burden of external memory devices bandwidth.Therefore, present embodiment is not to be written into again respectively to organize adjacent scanning lines for adjacent macro zone block, but adopt group (group) estimation mode, make in the present frame, the a group macro zone block that corresponds to the same central scan line (also namely, corresponding to the same vertical position of former frame) of fine lines buffer 14 is handled simultaneously.In other words, the decision of each macro zone block is relevant to the vertical MV component of macro zone block in the group.The estimation of Fig. 4 illustration group.As shown in Figure 4, three macro zone blocks of present frame are relevant to the same vertical component (as indivedual arrow indications) of former frame, therefore these three macro zone blocks are attributed to same group, according to the same scanning line set of fine lines buffer 14, to carry out estimation simultaneously.All finish dealing with when all macro zone blocks of same group, then capture another scanline groups and be stored in fine lines buffer 14.In a specific embodiment, only the macro zone block in the search area of present frame (search area as shown in Figure 4) is just handled, and therefore is able to the estimation of accelerated motion.Middle position that it should be noted that search area is determined by the central scan line.In other words, different central scan line will correspond to the middle position of different search areas.
According to above-described embodiment, the capacity of line buffer (12 and 14) will be reduced to SR* (1/N) * (1/M)+(2*N+1), and wherein SR is search area, and N is the reduction sampling factor of height, and M is the reduction sampling factor of width.Above-described embodiment can use hardware, software or its to make up to implement.Moreover embodiment also can use pipeline (pipelining) to implement.For example, the second stage of the hierarchical motion estimation of n frame can be carried out simultaneously with the phase I of the hierarchical motion estimation of (n+1) individual frame.
The above is the preferred embodiments of the present invention only, is not in order to limit claim of the present invention; All other do not break away from the equivalence of finishing under the disclosed spirit of invention and changes or modification, all should be included in the claims.
Claims (16)
1. the method for a hierarchical motion estimation comprises;
Reduce sampling one reference frame and a present frame;
Store this reduction sampling reference frame;
Reduce sampling reference frame and this reduction sampling present frame according to this, to produce a coarse movement vectogram;
Acquisition and storage are adjacent to the scan line of a central scan line, and this central scan line reduces one of sampling reference frame corresponding to this and reduces the sampling scan line; And
According to this coarse movement vectogram, this present frame with store scan line adjacent to this of this central scan line, to produce a fine movement vectogram.
2. the method for hierarchical motion estimation as claimed in claim 1, wherein a search area of this reference frame and this present frame is subjected to reducing sampling.
3. the method for hierarchical motion estimation as claimed in claim 1, wherein this reference frame is the former frame of leading this present frame.
4. the method for hierarchical motion estimation as claimed in claim 1, use one to reduce sampling factor N to the height do reduction sampling of this reference/present frame, use one to reduce sampling factor M to the width do reduction sampling of this reference/present frame, by this, every N pixel is chosen a pixel in vertical direction, every M pixel is chosen a pixel in the horizontal direction, and therefore, the size of this reference frame is reduced to (1/N) * (1/M).
5. the method for hierarchical motion estimation as claimed in claim 4, wherein the storage scan line adjacent to this central scan line comprises:
Be positioned at the N bar scan line on this central scan line; And
Be positioned at the N bar scan line under this central scan line;
By this, store (2*N+1) bar scan line altogether.
6. the method for hierarchical motion estimation as claimed in claim 1 in the step that produces this fine movement vectogram, in the present frame, corresponds to the macro zone block of this central scan line with processing simultaneously according to this coarse movement vectogram.
7. the method for hierarchical motion estimation as claimed in claim 6 wherein in this present frame, is handled the macro zone block that is positioned at a default search area.
8. the coarse movement vectogram that the method for hierarchical motion estimation as claimed in claim 1, the fine movement vectogram of n present frame produce step and (n+1) individual present frame produces step and carries out simultaneously.
9. the system of a hierarchical motion estimation comprises:
One first reduces sampling unit, in order to reduce sampling one reference frame;
One second reduces sampling unit, in order to reduce sampling one present frame;
One rough line buffer is used for storing this and reduces the sampling reference frame;
One coarse movement vector estimator is taken a sample present frame to produce a coarse movement vectogram according to this reduction sampling reference frame and this reduction;
One fine lines buffer, acquisition and storage are adjacent to the scan line of a central scan line, and this central scan line reduces one of sampling reference frame corresponding to this and reduces the sampling scan line; And
One fine movement vector estimator, according to this coarse movement vectogram, this present frame with store scan line adjacent to this of this central scan line, to produce a fine movement drawing for estimate.
10. the system of hierarchical motion estimation as claimed in claim 9, wherein a search area of this reference frame is subjected to this first reduction sampling that reduces sampling unit, and a search area of this present frame is subjected to this second reduction sampling that reduces sampling unit.
11. the system of hierarchical motion estimation as claimed in claim 9, wherein this reference frame is the former frame of leading this present frame.
12. the system of hierarchical motion estimation as claimed in claim 9, use one to reduce sampling factor N to the height do reduction sampling of this reference/present frame, use one to reduce sampling factor M to the width do reduction sampling of this reference/present frame, by this, every N pixel is chosen a pixel in vertical direction, every M pixel is chosen a pixel in the horizontal direction, and therefore, the size of this reference frame is reduced to (1/N) * (1/M).
13. the system of hierarchical motion estimation as claimed in claim 12, wherein the storage scan line adjacent to this central scan line comprises;
Be positioned at the N bar scan line on this central scan line; And
Be positioned at the N bar scan line under this central scan line;
By this, store (2*N+1) bar scan line altogether.
14. the system of hierarchical motion estimation as claimed in claim 9, wherein this fine movement vector estimator corresponds to the macro zone block of this central scan line with processing simultaneously according to this coarse movement vectogram in present frame.
15. the system of hierarchical motion estimation as claimed in claim 14, wherein this fine movement vector estimator is handled the macro zone block that is positioned at a default search area in present frame.
16. the system of hierarchical motion estimation as claimed in claim 9, wherein the fine movement vectogram of n present frame producing of this fine movement vector estimator and the coarse movement vectogram of (n+1) individual present frame that the vectorial estimator of this coarse movement produces are carried out simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010168223 CN102238380B (en) | 2010-04-22 | 2010-04-22 | Hierarchical motion estimation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010168223 CN102238380B (en) | 2010-04-22 | 2010-04-22 | Hierarchical motion estimation method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102238380A CN102238380A (en) | 2011-11-09 |
| CN102238380B true CN102238380B (en) | 2013-07-17 |
Family
ID=44888532
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010168223 Expired - Fee Related CN102238380B (en) | 2010-04-22 | 2010-04-22 | Hierarchical motion estimation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102238380B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780677B (en) * | 2016-12-15 | 2020-01-10 | 南京偶酷软件有限公司 | Method for simulating three-dimensional animation visual effect through lens motion background layering processing |
| CN108769682B (en) | 2018-06-20 | 2022-08-16 | 腾讯科技（深圳）有限公司 | Video encoding method, video decoding method, video encoding apparatus, video decoding apparatus, computer device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040032907A1 (en) * | 2002-08-13 | 2004-02-19 | Lowell Winger | System and method for direct motion vector prediction in bi-predictive video frames and fields |
| CN101010962A (en) * | 2004-08-31 | 2007-08-01 | 皇家飞利浦电子股份有限公司 | Method and device for motion estimation |
| CN101189870A (en) * | 2005-04-28 | 2008-05-28 | 德州仪器公司 | Motion stabilization |
| CN101355698A (en) * | 2008-06-10 | 2009-01-28 | 西南交通大学 | Frame for demixing and encoding stereo video |
| US20090086821A1 (en) * | 2007-10-01 | 2009-04-02 | Canon Kabushiki Kaisha | Image processing apparatus and method thereof |
-
2010
- 2010-04-22 CN CN 201010168223 patent/CN102238380B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040032907A1 (en) * | 2002-08-13 | 2004-02-19 | Lowell Winger | System and method for direct motion vector prediction in bi-predictive video frames and fields |
| CN101010962A (en) * | 2004-08-31 | 2007-08-01 | 皇家飞利浦电子股份有限公司 | Method and device for motion estimation |
| CN101189870A (en) * | 2005-04-28 | 2008-05-28 | 德州仪器公司 | Motion stabilization |
| US20090086821A1 (en) * | 2007-10-01 | 2009-04-02 | Canon Kabushiki Kaisha | Image processing apparatus and method thereof |
| CN101355698A (en) * | 2008-06-10 | 2009-01-28 | 西南交通大学 | Frame for demixing and encoding stereo video |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102238380A (en) | 2011-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105517671B (en) | Video frame interpolation method and system based on optical flow method | |
| US20120026367A1 (en) | System and method for maintaining maximum input rate while up-scaling an image vertically | |
| CN100563343C (en) | The method and apparatus that is used for estimation | |
| CN102271253A (en) | Image processing method using motion estimation and image processing apparatus | |
| CN103369208A (en) | Self-adaptive de-interlacing method and device | |
| US8692933B1 (en) | Method and apparatus for buffering anchor frames in motion compensation systems | |
| US8325815B2 (en) | Method and system of hierarchical motion estimation | |
| CN102238380B (en) | Hierarchical motion estimation method and system | |
| CN103826082B (en) | A kind of method for processing video frequency and device | |
| JP2011055278A (en) | Motion information acquiring apparatus and image processing device | |
| Zhang et al. | Optical flow reusing for high-efficiency space-time video super resolution | |
| US8644380B2 (en) | Integer pixel motion estimation system, motion estimation system for quarter-pixel luminance, motion estimation system for quarter-pixel chrominance, motion estimation system for combined luminance, motion estimation system for combined luminance and chrominance, and motion estimation system for quarter-pixel luminance and chrominance | |
| CN103780863B (en) | A kind of high-definition picture generation method and device | |
| US8837595B2 (en) | Motion estimation method | |
| US20090256968A1 (en) | Signal processing apparatus, and method for controlling signal processing apparatus | |
| US20110249188A1 (en) | Method of block-based motion estimation | |
| TWI404422B (en) | Method and system of hierarchical motion estimation | |
| CN1103090C (en) | Picture processing apparatus and processing method | |
| CN101860746B (en) | Motion estimation method | |
| US20090148067A1 (en) | Image processing method and related apparatus for performing image processing operation according to image blocks in horizontal direction | |
| JP2009296080A (en) | Super-resolution image generation system | |
| US20170324974A1 (en) | Image processing apparatus and image processing method thereof | |
| EP2105881A1 (en) | Fast reference frame selection for reconstruction of a high-resolution frame from low-resolution frames | |
| Wang et al. | P‐5.5: An Adaptive Video Scaling System Implementation Based on FPGA | |
| CN100471232C (en) | Digital dynamic image uninterleaving method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130717 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |