GB2309135A

GB2309135A - Estimating image motion by comparing adjacent image frame signals

Info

Publication number: GB2309135A
Application number: GB9700425A
Authority: GB
Inventors: Jechang Jeong; Byeungwoo Jeon; Wooyoun Ahn
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 1996-01-11
Filing date: 1997-01-10
Publication date: 1997-07-16
Anticipated expiration: 2017-01-10
Also published as: GB2309135B; MY121586A; GB9700425D0; KR970060161A; KR0185940B1

Abstract

A method for motion estimation includes steps for constructing a first image segment 31 representing a portion of present image frame and a second image segment 32 having a plurality of image segments of an adjacent frame, comparing 33 said plurality of image segments of said second image segment with the image signal within the first image segment, and generating a number of error values, detecting a first motion vector at one scale of resolution using the generated error values, outputting the error value associated with a reference segment pointed by the first motion vector, and the error values corresponding to the reference segment shifted by one unit of selected resolution respectively in selected direction(s), generating 34 a second motion vector in finer scale of resolution using the output error values, and summing 37 the first and second motion vectors.

Description

METHOD AND APPARATUS FOR NOTION ESTIMATION The present invention relates to a method and apparatus for performing motion estimation, and more particularly, though not exclusively relates to a method and apparatus for extremely detailed motion estimation which, for an estimation of motion vector(s) at a scale of finer resolution, uses error values generated at an estimation of motion for determining a motion vector, at a scale of lower resolution especially differential pulse code modulation (DPCM) method for coding a digital image signal.

Generally, there are several coding methods for further efficiently compressing transmission data in an image signal processing apparatus using subsequent digital image signals, such as high definition television (HDTV), Digital VTR (digital video tape recorder), and multimedia, etc. The DPCM method codes the digital image signal using data correlations which are present between adjacent frames of the image signal.

In the case of coding a difference signal between adjacent frames representing different times using the DPCM method, it is disadvantageous that much more data needs to be coded for an active image rather than a still image. However, by recognising a movement location from a certain segment of an adjacent frame to a specific segment in the present frame, that is, the best similarity between a specific image of the present frame and a certain image of the adjacent frame, efficient data compression can be performed because signal difference values of similar segments between frames can be minimised after motion compensation. Accordingly, in the motioncompensated DPCM method, difference data between a predetermined image segment of the present frame and the corresponding image segment of the adjacent frame is coded, thereby permitting efficiency of the transmission to be enhanced.In the motion estimation method, a motion vector represents a moving direction and magnitude of the image segment obtained by comparing an image segment of the present frame with the corresponding image segment of an adjacent frame.

There are several methods for generating motion estimates in terms of a pixel unit. Generally, since motion between two frames separated in time may be small with respect to an integer-pixel unit, errors may be introduced during motion compensation when using the motion vector of an integer-pixel unit. Therefore, a motion vector of a sub-pixel unit is used for reducing motion compensation errors. A half-pixel unit, as one estimation method of sub-pixel units, is commonly used for the motion estimation. A motion estimation method using a half-pixel unit in accordance with a conventional method will be briefly described.

The apparatus of Figure 1 comprises an image segment constructor 1 for receiving an image signal of the present frame and constructing an image segment composed of a plurality of pixels; search range constructor 2 for receiving the image signal of an adjacent frame and constructing the image segment composed of a plurality of pixels; a first search unit 3 for receiving the image signals output from the image segment constructor 1 and the search range constructor 2 and detecting a first motion vector MV1 of a pixel unit; a half-pixel interpolating unit 4 for receiving image signals output from the search range constructor 2 and the first search unit 3 and calculating image signal values in half-pixel locations around reference pixels, using the image segment, which will be referred to as a reference segment, within the adjacent frame which is determined by the first motion vector MV1 detected from the first search unit 3 and its surrounding pixels; a second search unit 5 for receiving the image signals output from the image segment constructor 1 and the half-pixel interpolating unit 4 and outputting a second motion vector MV2 of a half-pixel unit; and a summing unit 6 for receiving and summing the output signals MV1, MV2 from the first search unit 3 and the second search unit 5, respectively, and outputting a motion vector MV.

The image segment constructor 1 receives the image signal of the present frame and stores the input signal in segments. Simultaneous with inputting the image signal of the present frame, the search range constructor 2 receives the image signal from an adjacent frame and constructs a search range segment. The first search unit 3 searches for a segment within the search range segment which best matches the segment in the present form. By comparing data in a pixel unit between two segments the first motion vector MV1 of an integer-pixel unit generated from the first search unit 3 is output to the summing unit 6 and the half-pixel interpolating unit 4.The half-pixel interpolating unit 4 linearly interpolates out the reference pixels of a designated segment within the adjacent frame using the first motion vector MV1 of an integer-pixel unit and the surrounding pixels, and then outputs the calculated values to the second search unit 5.

In the above conventional system, the image segment may be a N1 X N2 block type composed of Nl X N2 pixels.

Additionally, the search range may be composed of Ml X M2 pixels. Alternately, the image segments and the search range may be constructed in accordance with the shape and size of the objects within the image. Furthermore, in the case of a uni-directional prediction which requires one adjacent frame for reference as in P-frame in MPEG or in B-frame that involves only backward prediction, the image segment of the present frame is compared with an image segment within the search range constructed either within the previous or future frame. However the motion estimation previously mentioned is also applicable to the bi-directional prediction which involves both previous and future frames.In the case of a bi-directional predictions as in B-frame in MPEG, the image segment of the present frame is compared with an image segment within the search ranges constructed within the previous frame and the future frame.

Another conventional method for constructing the image segment and the search range is accomplished in accordance with the objects within the image. Thus, constructed image segment and the search range segment may be of any size and shape.

In Figure 2, "O" illustrates pixel unit intervals and "x" indicates half-pixel unit intervals. The second unit 5 moves the image segment of an adjacent frame corresponding to the first motion vector MV1 of an integer-pixel unit to every direction in half-pixel units, respectively, searches for a location having a minimum motion compensation error among locations x for the obtained eight half-pixel motion vectors and the first motion vector MV1, and outputs one of the vector components (-1/2, 0, 1/2), that is a minutely controlled value in a half-pixel unit.The summing unit 6 sums the first motion vector MV1 in integer-pixel units output from the first search unit 3 and the second motion vector MV2 in half-pixel units from the second search unit 5, to thereby generate a complete motion vector MV for the motion estimation. For example, the summing unit 6 sums the horizontal component (-i/2) of the second motion vector MV2 of a half-pixel unit from the second search unit 5 with the motion vector (MVl=(x, y)) of a pixel unit from the first search unit 3, and then determines the motion vector (MV=(x-1/2, y)).

In the case of bi-directional motion-estimation, which involves previous and future adjacent frames, the same procedure is applied to both of forward and backward predictions.

The above-described conventional method determines the location of segment using the motion vector of a pixel interval unit, linearly interpolates out corresponding pixel values at half-pixel locations using the determined segment and surrounding pixels of the segment. The method uses the interpolated half-pixel values and the segment pixel values of the present frame determined by the motion vector in integer-pixel units, and selects the most similar segment among eight half-pixel motion positions, that is, "x" of Figure 2. Accordingly, the conventional method has a drawback in that it takes too long in processing data for estimating the motion vector in halfpixel units. This drawback is not confined to the case of motion vector estimation at half-pixel unit, but is a general problem for a motion vector estimation at a resolution finer than a pixel unit.

An aim of preferred embodiments of the present invention is to provide a method for motion estimation which makes efficient compensation of extremely detailed motion possible, wherein error values at one scale of resolution are used for estimation of motion vectors at a more detailed scale of resolution, comparing error values generated as a result of comparing data between predetermined image segment within the present frame and image segment within the search range to detect the motion vector in pixel unit, and estimating the motion vector at finer resolution.

Another aim of preferred embodiments of the present invention is to provide an apparatus for motion estimation making efficient compensation of extremely detailed motion possible, by employing error values used for estimation of motion vectors between frames, by generating error values corresponding to one-scale of resolution by comparing data between predetermined image segment within the present image segments with the adjacent frame(s), comparing generated errors, and estimating the motion vector at a finer scale.

According to the present invention in a first aspect, there is provided a method for comparing an image signal between adjacent frames and estimating a motion of the image, the method comprising the steps of: constructing a first image segment representing a portion of present image frame and a second image segment having a plurality of image segments of an adjacent frame; comparing said plurality of image segments of said second image segment with the image signal within the first image segment, and generating a number of error values; detecting a first motion vector at one scale of resolution using the generated error values; outputting the error value associated with a reference segment pointed by the first motion vector, and the error values corresponding to the reference segment shifted by one unit of selected resolution respectively in selected direction(s);; generating a second motion vector in a finer scale of resolution using the output error values; and summing the first and second motion vectors.

Suitably, said generating step further comprises the sub-steps of: detecting a component of said second motion vector in a first direction using the error value of said reference image segment and the error values of said surrounding image segments which are positioned in said first direction with respect to a centre of said reference image segment; and detecting a component of said section motion vector in a second direction using the error value of said reference image segment and the error values of said surrounding image segments which are positioned in said second direction with respect to the centre of said reference image segment.

Suitably, said first direction and said second direction are horizontal direction and vertical direction respectively.

Suitably, said component of said second motion vector in said first direction and said component of said second motion vector in said second direction indicate the location of the image segment corresponding to a minimum error value.

Suitably, said adjacent frame is a previous frame.

Suitably, said previous frame is an immediately previous frame with respect to said present frame.

Suitably, said previous frame is separated by a plurality of frames in time from said present frame.

Suitably, said adjacent frame is a future frame.

Suitably, said future frame is an immediately future frame with respect to said present frame.

Suitably, said future frame is separated by a plurality of frames in time from said present frame.

Suitably, each of said plurality of image segments of said second image segment has a segment size equal to said first image signal.

Suitably, said one scale of resolution is a one pixel-unit and said finer scale of resolution is a half pixel-unit.

According to the present invention in a second aspect, there is provided an apparatus for comparing an image signal between adjacent frames and estimating a motion of the image, the apparatus comprising: means for receiving the image signal of a present frame and constructing a first image segment; means for receiving the image signal of an adjacent frame and constructing a second image segment;; search means for receiving the image signal output respectively from the first image segment constructor and the second image segment constructor, detecting a first motion vector at one selected scale of resolution units, determining a reference segment within said second image segment in accordance with said first motion vector, comparing the image signals of surrounding image segments, which are generated by moving the reference image segment in selected direction(s) by one unit of the selected scale respectively, and the reference image segment with the image signal of the first image, and outputting error values obtained as a result of respective comparisons; control means for receiving the errors output from the search means and generating a second motion vector at a finer scale of resolution than the selected scale; and summing means receiving the first and second motion vectors, for adding and outputting them.

Suitably, said control means further comprises: a first controller for comparing said error value of said reference image segment with the error values of said surrounding image segments which are positioned in a first direction with respect to a centre of said reference image segment and detecting a vector component in said first direction of said second motion vector; and a second controller for comparing said error value of said reference image segment with the respective error values of said surrounding image segments which are positioned in a second direction with respect to the centre of said reference image segment and detecting a vector component in said second direction of said second motion vector.

Suitably, said controller further comprises: a first subtracter for subtracting the error value corresponding to said reference image segment from the error values corresponding to said surrounding image segments which are located to one side of said reference image segment; a second subtracter for subtracting the error value corresponding to said reference image segment from the error values corresponding to said surrounding image segments which are located to the other side of said reference image segment; a first multiplier for multiplying a respective first output signal of said second subtracter by a predetermined coefficient, and producing a first multiplied value; a second multiplier for multiplying a respective second output signal of said first subtracter by said predetermined coefficient, and producing a second multiplied value;; a first comparator, receiving said second output signal and said first multiplied value, for comparing respective magnitudes of the received first comparator inputs, thereby providing a first binary signal as a comparison result; a second comparator, receiving said first output signal and said second multiplied value, for comparing the respective magnitudes of the received second comparator inputs, thereby providing a second binary signal as a comparison result; and a NOR gate, receiving said first and second binary signals from said first and second comparators, respectively, for performing a logical NOR function, thereby producing an operated value indicative of said second motion vector.

Suitably, said adjacent frame is a previous fame.

Suitably, said adjacent frame is a future frame.

Suitably, said second image segment has a plurality of image segments.

Suitably, each of said plurality of image segments of said second image segment has a segment size equal to said first image segment.

According to the present invention there is provided a method for comparing an image signal between adjacent frames and estimating a motion of the image. The method comprises the steps of constructing a first image segment representing an image signal of a portion of present image frame and a second image segment which has the image signal of an adjacent frame and which will be referred to as a search range segment, comparing the image signal of a plurality of image segments which are present within the search range segment with the image signal within the first image segment, and generating a number of error values; outputting the error value associated with the segment pointed by the first motion vector which will be referred to as a reference segments, and the error values corresponding to the reference segment shifted by one unit of selected resolution respectively in selection direction(s); generating a second motion vector in finer scale of resolution using the output error values; and summing the first and second motion vectors.

According to the present invention there is also provided an apparatus for comparing an image signal between adjacent frames and estimating a motion of the image. The apparatus comprises means for receiving the image signal of a present frame and constructing a first image segment; means for receiving the image signal of an adjacent frame and constructing a second image segment;; search means for receiving the image signal output respectively from the first image segment constructor and the second image segment constructor, detecting a first motion vector at selected scale of resolution units, comparing the image signals of surrounding image segments, which are generated by moving the reference image segment within the adjacent frame pointed by the detected first motion vector in selected direction(s) by one unit of the selected scale respectively, and the reference image segment, with the image signal of the first image, and outputting error values obtained as a result of respective comparisons; control means for receiving the errors output from the search means and generating a second motion vector having a resolution finer than the selected scale; and summing means receiving the first and second motion vectors, for adding and outputting them.

Preferred embodiments of the present invention are described, by way of example only, with reference to the drawings wherein: Figure 1 is a block diagram for illustrating a conventional apparatus used for half-pixel motion estimation.

Figure 2 is a conceptual diagram for showing a pixel unit and a half-pixel unit.

Figure 3A is a block diagram illustrating an apparatus for performing motion estimation in accordance with a preferred embodiment of the present invention.

Figure 3B is another example of a block diagram illustrating an apparatus for performing motion estimation in accordance with a preferred embodiment of the present invention.

Figure 4 is a detailed block diagram illustrating a horizontal half-pixel controller of Figures 3A-3B.

Figures 5(a) and 5(b) are conceptual diagrams showing a method for determining motion vectors of a half-pixel unit in the horizontal half-pixel controller.

Description is now made for a case in which a first motion vector selected at any scale and a second motion vector at a finer scale are selected to be pixel unit resolution and half-pixel unit respectively. This example however, can be applied to any other selected scales.

A motion vector of a predetermined segment in a present frame corresponding to a segment in an adjacent frame is represented as MV= (x,y) . Here, the motion vector MV has a horizontal component of motion vector x and a vertical component of motion vector y.

Figure 3A is a block diagram illustrating an apparatus for motion estimation in accordance with a preferred embodiment of the present invention.

The apparatus of Figure 3A comprises an image segment constructor 31 for receiving an image signal of the present frame and constructing image segments composed of a plurality of pixels; search range constructor 32 for receiving the image signal of an adjacent frame and constructing image segments composed of a plurality of pixels; search unit 33 for receiving the image signals output from the image segment constructor 31 and the search range constructor 32, for detecting a first motion vector MV1 in pixel units by comparing image signals between image segments within the segment constructed by the search range constructor 32 and the image segment in the present frame, and outputting error values corresponding to an image segment pointed by the detected MV1, which will be called as a reference segment and is generated by moving the reference segment in horizontal and vertical directions by one pixel; half-pixel control means 34 for receiving the errors output from the search unit 33 and calculating a second motion vector MV2 in respective half-pixel interval vertically and horizontally; and summing unit 37 receiving the first motion vector MV1 from the search unit 33 and the second motion vector MV2 from the half-pixel control means 34, summing the respective vectors and outputting a resultant motion vector MV.

The half-pixel control means 34 includes a horizontal half-pixel controller 35 for detecting a horizontal component of the second motion vector MV2 and vertical half-pixel controller 36 for detecting a vertical component of the second motion vector MV2.

Of the image signals for two adjacent frames separated in time, the image segment constructor 31 receives the image signal of the present frame and stores the input signal in a first segment. Simultaneously with inputting the image signal of the present frame, the search range constructor 32 receives the image signal of an adjacent frame and stores it in a second segment, which advantageously may be larger than the first segment. The search unit 33 receives respectively the image signals of the stored second segment and the first segment output from the search range constructor 32 and the image segment constructor 31, respectively.In the case of a P-frame, the comparison is made between the present frame and a previous frame; however in the case of a B-frame the comparison is made between the present frame and a future frame in addition to the comparison between the present frame and a previous frame. Thus, in the case of a Bframe, the search range constructor 32 constructs a second segment of a previous frame and a third segment of a future frame. Accordingly, the comparison between the present frame and the previous frame and then between present frame and future frame or vice versa or the comparisons may occur simultaneously as shown in Figure 3B.

The following is a description in which the first segment and the second segment have the form N1 X N2 pixels and Ml X M2 respectively. Furthermore, for simplicity of explanation, an integer-pixel unit is selected as one scale of resolution and a half-pixel unit is selected as the finer scale of resolution. Note that the present invention can be applied to the cases with other selected scale resolutions as well.

A motion estimation error obtained by estimating the motion vector of an integer-pixel unit is represented by P0, Motion estimation errors which are generated by moving in horizontal and vertical direction by one pixel, respectively, on the basis of the estimated vector, are respectively Pus', Pl', P.1 and P1. Mean absolute error MAE, or mean squared error MSE advantageously can be used for calculating motion estimation errors. Equations which preferably can be used to calculate the horizontal motion estimation errors (PO, Pl, P) by the MAE are as follows:

Where Y represents the image signal of the present frame, Y' represents that of the adjacent frame, and Nl=N2=N represent segment size.In the above equations, (k, 1) represents position from the top left position of the frames. It will be appreciated that the vertical motion estimation errors (P1,, P.') can be calculated by the same method as noted by the above listed equation.

The search unit 33 compares, in pixel units, the data magnitude between the image segment of the present frame and a plurality of adjacent image segments, which are present within the search range of the adjacent frame and have similar sizes, and calculates motion estimation errors as a result of respective comparisons. The first motion vector MV1, in integer-pixel units is determined as a segment location where the motion estimation error is a minimum, and is output to the summing unit 37.The search unit 33 outputs to the half-pixel control unit 34 the motion estimation errors (Pl, PX, P0, Pl, Pl) corresponding to the reference image segment within an adjacent frame determined by the motion vector MV1 in integer-pixel units and a number of image segments surrounding the reference image segment and shifted by one pixel. Advantageously, the present embodiment simultaneously and individually determines the horizontal and vertical components of motion vector in half-pixel units. However, since the horizontal half-pixel controller 35 and the vertical half-pixel controller 36 use the same methodology, description will be limited to a method for determining the horizontal component of a motion vector in half-pixel units.

In the case of the uni-directional motion prediction, a single set of motion estimation errors (pal, P, P0, Pal', resulting from comparison of the present frame and adjacent frame is output from the search unit 33 to the half-pixel control means 34 as described above; however, the same argument can be made to the bi-directional motion prediction as in B-frame, two sets of motion errors, one set resulting from comparison of the frame and a previous frame and another set resulting from of the present frame and a future frame, are output from search unit 33 to the half-pixel control unit 34.

Figure 3B is similar to Figure 3A except that two adjacent frames are simultaneously input to the system.

In the case of a B-frame, a future frame and a previous frame are simultaneously input to the system.

Figure 4 is a detailed block diagram illustrating the horizontal half-pixel controllers of Figures 3A and 3B.

The device of Figure 4 includes three input terminals 41, 42, 43 for receiving the motion estimation errors (P1, P0, Pal); a first adder Al for adding and outputting the input signal P1 of a first input terminal 41 and the input signal P0 of a second input terminal 42; a second adder A2 for adding and outputting the input signal P0 of the second input terminal 42 and the input signal PX of a third input terminal 43; a first comparator CMP1, having a predetermined input terminal coupled to an output terminal of the first adder Al, for comparing respective input signals; a second comparator CMP2, having a predetermined input terminal coupled to an output terminal of the second adder A2, for comparing respective input signals; a second multiplier M2 for multiplying the output signal of the first adder Al by a predetermined coefficient and outputting a multiplied value to the second comparator CMP2; a first multiplier M1 for multiplying the output signal of the second adder A2 by a predetermined coefficient and outputting a multiplied value to the first comparator CMP1; and a NOR gate NOR for performing a logical NOT OR function on respective output signals from the comparators CMP1, CMP2. The device further comprises three output terminals 44, 45, 46 to distinguish each of the output signals from first comparator CMP1, NOR gate NOR and second comparator CMP2.

The motion estimation errors (Pl, P0, P) output from the search unit 33 are input to the horizontal half-pixel controller 35 of the half-pixel control means 34. Then, the first adder Al subtracts the signal P0 input through the second terminal 42 input from the input signal P1 through the input first terminal 41. The second adder A2 subtracts the signal P0 input through the second input terminal 42 from the signal P.1 input through the third input terminal 43. The first and second multipliers M1, M2 multiply by respective predetermined coefficient each of the adder output signals and output the multiplied result to the comparators CMP1, CMP2, connected to the output of the multipliers, respectively.Advantageously, a difference value (a) between P0 and PX and (b) between P0 and P1 are respectively input to the second comparator CMP2 and the first comparator CMP1. Preferably, the first comparator CMP1 compares and outputs the output signals from the first adder Al and the first multiplier Ml. The second comparator CMP2 compares and outputs the output signals from the second adder A2 and the second multiplier M2.

Figures 5(a) and 5(b) are conceptual diagrams showing a method for determining motion vectors in half-pixel units in the horizontal half-pixel controller 35. Figure 5(a) shows that the motion vector MV2 of a half-pixel unit is present at the left with respect to the motion vector MV1 of a pixel unit, and Figure 5(b) shows that the motion vector MV of a half-pixel unit is present at the right with the motion vector MV1 of a pixel unit.

In Figure 5(a), when P1 is larger than P1 (that is, b is larger than a) it indicates that the motion vector MV is located about 1/2 the distance from the first motion vector MV1 in pixel units to the left. In Figure 5(b), when P1 is larger than P1 on the basis of P0 (that is, a is larger than b) indicates that the motion vector MV is located about 1/2 way from the first motion vector MV1, in pixel units to the right.

In Figure 5(a), for the case where P1 is larger than P.1 on the basis of P0, only the output 44 of the first comparator CMPl is "high". Thus, the horizontal halfpixel controller 35 outputs "-1/2", as the horizontal component value of the second motion vector MV2. In Figure 5(b), for the case where P.1 is larger than P1 on the basis of P0, only the output 46 of the second comparator CMP2 is "high". Thus the horizontal half-pixel controller 35 outputs "1/2", as the horizontal component value of the second motion vector MV2. Where P1 and P1 on the basis of P0 are equal to each other the output signals of the two comparators CMP1, CMP2 are both "low".Therefore, the NOR gate NOR receives low values and its output signal becomes "high". Accordingly, the horizontal component of the second motion vector MV2 becomes "0".

The summing unit 37 receives the second motion vector MV2 and adds it to the first motion vector MV1, in pixel units, obtained by the search unit 33, thereby estimating the second motion vector MV2 in half-pixel units.

It will be appreciated that a method and an apparatus for motion estimation in accordance with preferred embodiments of the present invention compares image signals of segments within an adjacent frame with the selected segment in a present frame to detect a motion vector at a selected scale of resolution; generates error values calculated in the detection of the motion vectors according to the selected motion vector.

By vertically comparing the generated error values, a vertical component of the motion vector in the finer scale of resolution is detected. By horizontally comparing the generated error values, a horizontal component of the motion vector in the finer scale of resolution is detected. Accordingly, preferred embodiments of the present invention lead to a greater efficiency in motion estimation by reducing the amount of data which must be processed for the detection of the motion vector in a finer scale of resolution.

Although the preferred embodiments of the present invention have been described with the image segment and the search range having a particular form, the image segment and the search range may be of any shape or size such as in the case in which the image segment and the search range is constructed in accordance with the shapes and sizes of the objects. Furthermore, the comparison of a segment from the present frame may be made with an immediately previous frame or immediately future frame as well as previous or future frames separated from the present frame by more than one frame in time.

Furthermore, the comparison may be made to any combination of above mentioned frames such as B-frame used in the MPEG system in which more than one adjacent frame are compared.

While only certain embodiments of the invention have been specifically described herein, it will be apparent that numerous modifications may be made thereto without departing from the spirit and scope of the invention.

The reader's attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.

All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

The invention is not restricted to the details of the foregoing embodiment(s). The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed.

Claims

Claims:

1. A method for comparing an image signal between adjacent frames and estimating a motion of the image, the method comprising the steps of: constructing a first image segment representing a portion of present image frame and a second image segment having a plurality of image segments of an adjacent frame; comparing said plurality of image segments of said second image segment with the image signal within the first image segment, and generating a number of error values; detecting a first motion vector at one scale of resolution using the generated error values; outputting the error value associated with a reference segment pointed by the first motion vector, and the error values corresponding to the reference segment shifted by one unit of selected resolution respectively in selected direction(s);; generating a second motion vector in a finer scale of resolution using the output error values; and summing the first and second motion vectors.

2. A method as claimed in claim 1, wherein said generating step further comprises the sub-steps of: detecting a component of said second motion vector in a first direction using the error value of said reference image segment and the error values of said surrounding image segments which are positioned in said first direction with respect to a centre of said reference image segment; and detecting a component of said second motion vector in a second direction using the error value of said reference image segment and the error values of said surrounding image segments which are positioned in said second direction with respect to the centre of said reference image segment.

3. A method as claimed in claim 2 wherein said first direction and said second direction are horizontal direction and vertical direction respectively.

4. A method as claimed in claim 2 or claim 3 wherein said component of said second motion vector in said first direction and said component of said second motion vector in said second direction indicate the location of the image segment corresponding to a minimum error value.

5. A method as claimed in any preceding claim wherein said adjacent frame is a previous frame.

6. A method as claimed in claim 5 wherein said previous frame is an immediately previous frame with respect to said present frame.

7. A method as claimed in claim 5 wherein said previous frame is separated by a plurality of frames in time from said present frame.

8. A method as claimed in any one of claims 1-4 wherein said adjacent frame is a future frame.

9. A method as claimed in claim 8 wherein said future frame is an immediately future frame with respect to said present frame.

10. A method as claimed in claim 8 wherein said future frame is separated by a plurality of frames in time from said present frame.

11. A method as claimed in any preceding claim wherein each of said plurality of image segments of said second image segment has a segment size equal to said first image segment.

12. A method as claimed in any preceding claim wherein said one scale of resolution is a one pixel-unit and said finer scale of resolution is a half pixel-unit.

13. An apparatus for comparing an image signal between adjacent frames and estimating a motion of the image, the apparatus comprising: means for receiving the image signal of a present frame and constructing a first image segment; means for receiving the image signal of an adjacent frame and constructing a second image segment;; search means for receiving the image signal output respectively from the first image segment constructor and the second image segment constructor, detecting a first motion vector at one selected scale of resolution units, determining a reference segment within said second image segment in accordance with said first motion vector, comparing the image signals of surrounding image segments, which are generated by moving the reference image segment in selected direction(s) by one unit of the selected scale respectively, and the reference image segment, with the image signal of the first image, and outputting error values obtained as a result of respective comparisons; control means for receiving the errors output from the search means and generating a second motion vector at a finer scale of resolution than the selected scale; and summing means receiving the first and second motion vectors, for adding and outputting them.

14. An apparatus as claimed in claim 13, wherein said control means further comprises: a first controller for comparing said error value of said reference image segment with the error values of said surrounding image segments which are positioned in a first direction with respect to a centre of said reference image segment and detecting a vector component in said first direction of said second motion vector; and a second controller for comparing said error value of said reference image segment with the respective error values of said surrounding image segments which are positioned in a second direction with respect to the centre of said reference image segment and detecting a vector component in said second direction of said second motion vector.

15. An apparatus as claimed in claim 14, wherein said first direction and said second direction are horizontal direction and vertical direction respectively.

16. An apparatus as claimed in claim 14 or claim 15, wherein said controller further comprises: a first subtracter for subtracting the error value corresponding to said reference image segment from the error values corresponding to said surrounding image segments which are located to one side of said reference image segment; a second subtracter for subtracting the error value corresponding to said reference image segment from the error values corresponding to said surrounding image segments which are located to the other side of said reference image segment; a first multiplier for multiplying a respective first output signal of said second subtracter by a predetermined coefficient, and producing a first multiplied value;; a second multiplier for multiplying a respective second output signal of said first subtracter by said predetermined coefficient, and producing a second multiplied value; a first comparator, receiving said second output signal and said first multiplied value, for comparing respective magnitudes of the received first comparator inputs, thereby providing a first binary signal as a comparison result; and a second comparator receiving said first output signal and said second multiplied value, for comparing the respective magnitude of the received second comparator inputs, thereby providing a second binary signal as a comparison result; and a NOR gate, receiving said first and second binary signals from said first and second comparators, respectively, for performing a logical NOR function, thereby producing an operated value indicative of said second motion vector.

17. An apparatus as claimed in any one of claims 13-16 wherein said adjacent frame is a previous frame.

18. An apparatus as claimed in claim 17 wherein said previous frame is an immediately previous frame with respect to said present frame.

19. An apparatus as claimed in claim 17 wherein said previous frame is separated by a plurality of frames in time from said present frame.

20. An apparatus as claimed in any one of claims 13-16 wherein said adjacent frame is a future frame.

21. An apparatus as claimed in claim 20 wherein said future frame is an immediately future frame with respect to said present frame.

22. An apparatus as claimed in claim 20 wherein said future frame is separated by a plurality of frames in time from said present frame.

23. An apparatus as claimed in any one of claims 13-22 wherein said second image segment has a plurality of image segments.

24. An apparatus as claimed in claim 23 wherein each of said plurality of image segments of said second image segment has a segment size equal to said first image segment.

25. An apparatus as claimed in any one of claims 13-24 wherein said one scale of resolution is a one pixel-unit and said finer scale of resolution is a half pixel-unit.

26. A method for motion estimation substantially as described herein.

27. An apparatus for motion estimation substantially as described herein, with reference to and as shown in Figures 3A and 4 or 3B and 4 of the accompanying drawings.